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EXEXin.1VE  SllMARY 

Tsss  CASE  srocv:  EsmeriNG  the  oqgt  op 
ADA  SCffTWRE  CEVElTXffQn' 


The  Ada  Joint  Program  Office  '(AJPO)  has  had  a  number  of  inquiries 
on  hew  to  estimate  the  cost  of  Ada  projects,  and  specifically  whether 
current  software  cost  models  that  are  non-Ada  specific  can  be 
successfully  used  to  estimate  the  cost  of  Ada  develcpnent  efforts.  No 
independent  data  currently  exists  that  addresses  the  fidelity  of  cost 
models  to  predict  Ada  softwcure  costs  accurately. 

IIT  Research  Institute  -(IITRI)  has  performed  a  study  for  the  U.S. 
Air  Force  Cost  Center '^(APCSriC)  'and  the  U.S.  Array  Cost  and  Economic 
Analysis  Center'  (USACEAC) ,  under  sponsorship  of  the  AJPO,  to  assess  the 
accuracy  of  current  cost  models  for  Ada  software  cost  estimation.  The 
study  focussed  on  six  cost  estimation  models  and  their  philosophies 
regarding  Ada.  The  bases  for  these  models  present  different  views  of 
the  model  develc^^ers  relative  to  the  costing  issues. 

The  guiding  principle  for  model  selection  for  inclusion  in  the 
study  was  the  availability  of  models  to  the  AFCSTC,  USACEAC,  and  IITRI. 
Models  reviewed  were  cis  follows: 

Ada-Specific  Models  Non-Ada-Specific  Models 

1.  Ada  COOCM)  (IOC)  1.  HUGE  S  (1882?0) 

2.  SoftCost-Ada  (1.3)  2.  SYSTEM-3  (1.03) 

3.  SPQIV20  (1.2) 

4.  SASET  (1.5) 

The  version  of  each  model  reviewed  is  indicated  in  parenthesis  beside 
the  model  name. 

An  essential  part  of  the  resectrch  is  a  test  case  study  in  vhich 
the  cost  models  were  applied  to  a  database  of  eight  conpleted  Ada 
projects.  Project  questionnaires  were  ccjipleted  by  the  developers. 
This  information  was  used  to  derive  inputs  for  each  model.  Enphasis 
was  placed  on  providing  a  consistent  set  of  inputs  across  all  models. 
In  addition,  models  were  applied  using  nominal  (average)  values  for 
input  ratings  while  providing  actual  project  values  for  size, 
application  type,  programming  language,  and  other  model  iiputs  that 
must  be  estimated  early  in  the  life  cycle  and  for  vhich  there  is  no 
associated  average  value.  The  nominal  inputs  reflect  the  level  of 
knowledge  about  a  new  project  prior  to  contract  award.  Resultant 
analyses  were  beised  on  a  carpeurison  of  each  model's  schedule  and  effort 
projection  cind  nominal  run  results  to  the  actual  project  resources 
expended  by  the  software  developer. 
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Results  were  evaluated  for  acxxiracy  and  consistency  for  each  of 
the  follcwing  six  categories: 

1 .  Overcill  effort 

2 .  Overall  schedule 

3 .  Government  contracts 

4 .  Canmercial  contracts 

5.  Canmand  and  control  applications 

6.  Tools  and  environment  applications. 

Tcibles  1  and  2  respectively  suitttarize  the  test  ccise  study  results  for 
derived  and  ncminal  inputs.  For  each  evaluation  criteria,  the  two 
models  that  had  the  hic^est  performance  ratings  are  listed.  Model 
results  were  also  evaluated  based  on  the  project's  design  approach  and 
personnel  experience  with  Ada.  Model  performances  could  not  be 
correlated  to  the  lauiguage  considerations  of  the  models. 

Ihe  test  case  study  results  demonstrate  the  benefits  of  cost 
models  that  assist  the  estimator  in  predicting  resource  requirements 
for  a  new  development.  Ihe  results  do  not  vadidate  the  need  for  Ada- 
specific  models.  Althou^  SoftCost-Ada  vas  most  accurate  overall,  non- 
Ada-specific  models  were  ccmpeirable  in  terms  of  accuracy  and 
consistency. 

The  results  do  recommend  that  users  consider  the  following  to 
determine  which  models  should  be  applied  to  estimate  Ada  software 
costs: 


Assess  how  much  informaticn  is  a[vailable  about  the  project 
cind  the  developing  organization.  Application  of  models  to 
both  regulcur  and  ncmineil  runs  in  the  study  shewed  that  model 
performances  varied  with  differing  amount  of  project 
information.  Some  performed  better  with  minimum  information 
v^hile  others  performed  better  with  detailed  information. 

Consider  the  customer.  Model  performances  were  eveduated 
based  on  the  type  of  contract.  Some  models  were  more 
effective  vhen  applied  to  government  contracts  while  others 
were  more  accurate  for  estimating  commercial  contracts. 

nonsider  the  type  of  a^lication.  Projects  targeted  in  the 
Lest  case  study  consisted  of  three  different  types  of 
applications:  command  and  control  (4  projects), 

'  ^jls/environment  (3  projects) ,  and  avionics  (1  project) .  An 
ancilysis  of  results  based  on  application  typse  reve2Lled  that 
models  that  were  most  accurate  on  oommeind  and  control 
applications  were  not  as  accurate  for  tools  and  environment 
applications. 
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TABLE  1 


I 
I 

I  SUMMARY  TEST  CASE  STUDY  RESULTS: 

BEST  TWO  PERPOPMANCES  IN  EACH  CATEGORY 

I 


Evaluaticn 

Criteria 

Model 

Perfarmanoe 
(Within  30%) 

Range 

Overall  Accuracy 
of  Effort 

SoftCost-Ada 

SASET 

4  out  of  7 

4  out  of  8 

0%  to  13% 
-29%  to  -29% 

Overall  Accuracy 
of  Schedule 

SYSTEM-3 

PRICE  S 

4  out  of  8 

3  out  of  8 

-27%  to  -  7% 
3%  to  18% 

Overall  Consistency 
of  Effort 

SYSTEM-3 

PRICE  S 

5  out  of  8 

5  out  of  8 

-14%  to  28% 
-26%  to  22% 

Overall  Consistency 
of  Schedule 

SYSTEM-3 

PRICE  S 

5  out  of  8 

5  out  of  8 

0%  to  28% 
-29%  to  28% 

Model  Accuracy  on 
Govemment  Contracts 

SASET 

OOSTMODL 

3  out  of  4 

2  out  of  4 

-  7%  to  29% 
-25%  to  -  1% 

Model  Consistency  on 
Government  Contracts 

SYSTEM-3 

PRICE  S 

. 

4  out  of  4 

3  out  of  4 

-14%  to  28% 
-26%  to  0% 

Model  Accuracy  on 
Commercial  Contracts 

SoftCost-Ada 

SPQiV20 

3  out  of  3 

1  out  of  4 

0%  to  6% 
-22% 

Model  Consistency  on 
Commercial  Contracts 

SoftCost-Ada 

PRICE-S 

3  out  of  3 

2  out  of  4 

-13%  to  -  8% 

-  1%  to  22% 

Model  Accuracy  on 
command  &  Control 
;^plications 

SASET 

SPQFV20 

3  out  of  4 

3  out  of  4 

-  7%  to  29% 
-22%  to  19% 

Model  Consistency  on 
ComiTand  &  Control 
Applications 

PRICE  S 

SASET 

4  out  of  4 

3  out  of  4 

-26%  to  0% 
-15%  to  1% 

Model  Accuracy  on 
Tools/Environment 
Applications 

SoftCost-Ada 

SASET 

2  out  of  2 

1  out  of  3 

0%  to  2% 
-29% 

Model  Consistency  on 

Tools/Environment 

Applications 

SoftCost-Ada 

SYSTEM-3 

2  cut  of  2 

1  out  of  3 

-13%  to  -11% 
28% 

IX 


TABIE  2 


SUM®RY  TEST  CASE  STUDY  RESUITS  FDR  NCMINAL  RUNS: 
BEST  TWO  PERFDEMANCES  IN  EACH  CATEGORY 


Evaliiaticn 

OrLterda 

Model 

Perfomanoe 
(Within  30%) 

Range 

Overall  Acxxiracy 
of  Effort 

SASET 

SYSTEM-3 

4  out  of  8 

3  out  of  8 

-24%  to  29% 
-17%  to  28% 

Overall  Accuracy 
of  Schedule 

SPQIV20 

FRIGE  S 

6  out  of  8 

4  out  of  8 

-23%  to  28% 
-26%  to  21% 

Overall  Consistency 
of  Effort 

COSTMODL 

SoftCost-Ada 

3  out  of  6 

3  out  of  7 

-23%  to  30% 
0%  to  28% 

Overall  Consistency 
of  Schedule 

SF>QFV20 

FRIGE  S 

6  out  of  8 

5  out  of  8 

-28%  to  20% 
-28%  to  29% 

Model  Accuracy  on 
Government  Contracts 

SASET 

FRIGE  S 

3  out  of  4 

2  out  of  4 

-  7%  to  29% 
-14%  to  -  8% 

Model  Consistency  on 
Government  Contracts 

SoftCost-Ada 

SYSTEM-3 

3  out  of  4 

3  out  of  4 

0%  to  28% 
-26%  to  13% 

Model  Accuracy  on 
Commercial  Contracts 

COSTMODL 

SoftCost-Ada 

1  out  of  2 

1  out  of  3 

-24% 

14% 

Model  Consistency  on 
Commercial  Contracts 

SASET 

SF>QB/20 

1  out  of  4 

1  out  of  4 

7% 

-14% 

Model  Accuracy  on 
Command  &  Control 
Applications 

SASET 

SYSTEM-3 

3  out  of  4 

3  out  of  4 

-  7%  to  29% 
-17%  to  28% 

Model  Consistency  on 
Command  &  Control 
Applications 

SASET 

SoftCost-Ada 

3  out  of  4 

2  out  of  4 

-24%  to  7% 
12%  to  28% 

Model  Accuracy  on 
Tools/Environment 
Applications 

SASET 

1  out  of  3 

-24% 

Model  Consistency  on 

Tools/Environment 

i^lications 

0 

1.0  nnKxxJcncN 


1.1  BACRC3CTJND 

Ihe  transition  to  a  new  technology  like  Ada  takes  considerable 
time,  conmitment,  and  resources.  It  is  generally  held  that  during  the 
short-term,  Ada  software  projects  will  cost  more  as  a  result  of 
inexperienced  staff,  immature  software  support  tools,  and  design  and 
packaging  costs  for  developing  reusable  softwcire.  These  additional 
costs  will  be  offset  in  the  long-term  as  developers  begin  to  derive  the 
benefits  of  reuse  and  personnel  spend  less  effort  maintaining  code  that 
is  less  error  prone. 

Perc^jtions  vairy  widely  on  how  to  estimate  the  cost  of  Ada 
projects.  Since  Ada  was  established  as  the  single  cannon  corputer 
progranming  language  for  all  new  developments  and  major  upgrades  of 
Defense  systems,  two  Ada-specific  cost  estimating  models  have  been 
develcped;  SoftCost-Ada  and  Ada  0CX3CMD.  In  addition,  cost  drivers  in 
several  other  models  have  been  adjusted  so  that  they  could  sipport 
near-term  and  long-term  Ada  cost  estimating  needs. 

The  bcises  for  these  models  present  the  different  views  of  the 
model  developers  relative  to  costing  issues.  An  Ada  study  conducted  by 
Reifer  Consultants,  Inc.  (RCI) ,  developers  of  SoftOost-Ada,  during  1986 
[REIF87]  concluded  that  existing  software  cost  models,  largely  based  on 
non-Ada  development  efforts,  do  not  accurately  account  for  Ada's  risks 
and  actual  experiences.  Fewer  laws  for  Ada  are  different  from  other 
high  order  languages  (HOLs)  and  their  effects  cannot  be  masked  by 
adjusting  cost  estimating  relationships.  Calibration  is  different 
becaiise  of  the  need  to  account  for  the  Ada  learning  curve  and  software 
reuse. 


Dr.  Barry  Boehm  of  TRW,  Inc.,  a  co-developer  of  Ada  ODCCMO,  agrees 
that  Ada  and  the  process  for  developing  Ada  software  irpacts  the  manner 
in  vhich  development  cost  is  estimated.  While  the  existing  COOQMD  has 
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done  reascnably  well  in  estinating  Ma  software  costs,  e^q^erienoe  to 
date  at  T3W  has  indicated  that  a  version  of  GOCCMD  specifically 
tailored  to  Ada  would  probably  do  better.  Ihe  effects  of  using  Ada  and 
practices  relating  to  the  Ada  Process  Model  lead  to  new  cost  estimating 
relationships  that  acooranodate  the  resulting  cost  and  schedule  effects. 

Other  ind^)endent  studies  conducted  by  developers  of  SFQIV20, 
SYSTEM-3,  and  PRICE  S  reccmnend  no  significant  modifications.  These 
studies  indicate  that  present  models  are  adequate  for  Ada  estimation 
with  the  provision  that  some  input  parameters  are  sli<^tly  modified 
based  on  Ada  experience.  User's  of  these  models  select  their  inputs 
with  regard  to  a  ceilibrated  baseline  that  represents  their  tYpic2d 
development.  The  input  responses  reflect  the  ways  in  ^diich  the  Ada 
project  differs  frcm  a  reference  baseline.  Learning  curves  and  reuse 
factors  are  not  treated  as  Ada-unique  phenomena. 

The  Ada  Joint  Program  Office  (AJPO)  has  had  a  number  of  inquiries 
on  how  to  estimate  the  cost  of  an  Ada  project  and  vdiether  current 
software  cost  models  that  are  non-Ada  specific  can  be  successfully  used 
to  estimate  the  cost  of  Ada  develc^ment  efforts.  Currently,  there  is 
no  independent  data  that  validates  the  need  for  Ada^-specific  models  or 
justifies  the  use  of  present  models  that  are  non-Ada-specific.  Part  of 
the  problem  in  evaluating  the  accuracy  of  a  model  (conparing  the 
model's  projections  to  the  actual  cost  of  a  development)  is  that  many 
contracts  are  awarded  at  a  firm  fixed  price.  Thus,  unless  a  concerted 
effort  is  made  to  track  the  real  cost  and  scope  of  the  effort,  no 
assessment  can  be  made  with  regard  to  the  fidelity  of  the  model  to 
predict  Ada  software  costs  accurately. 

1.2  OBJEXinVE 

The  objectives  of  this  report  are  centered  euxjund  the  major  issues 
regarding  Ada  software  development  cost  estimation.  To  assist  the  cost 
analyst  in  arriving  at  reasonable  Ada  cost  estimates,  this  r^xjrt  will 
focus  on  the  following  objectives: 
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(1)  Determine  if  software  ocjst  irodels  that  are  non-Ada-^jecific 
can  be  successfully  used  to  estimate  the  cost  of  Ada 
developnent. 

(2)  Provide  a  ccnpr^iensive  review  of  selected  cost  models  with 
regard  to  Ada-specific  cost  drivers  and  issues  discussed  in 
literature.  Evaluate  model  performance  in  light  of  the 
language  oonsideraticais  of  the  models  to  determine  any 
correlation. 

(3)  Identic  which  models  provide  reasonable  results  when  used  to 
estimate  Ada  develcpment  cost. 


An  essential  part  of  the  resecuxh  is  a  test  ceise  study  in  which 
selected  cost  models  were  applied  to  a  database  of  ei^t  ocnpleted  Ada 
projects.  This  information  was  used  to  derive  inputs  for  each  model. 
Enphasis  was  placed  on  providing  a  consistent  set  of  information  for 
each  model.  In  addition,  models  were  applied  using  ncrdnal  (average) 
values  for  input  ratings  vhile  providing  actral  project  veilues  for 
model  irputs  vhich  have  no  associated  average  value.  Model  performance 
was  evaluated  based  on  a  cotparison  of  each  model's  projected  schedule 
and  effort  to  the  actual  resources  expended  by  the  software  developer 
for  each  project.  Ihe  results  of  the  test  case  study  are  the  bases  for 
recommending  a  preferred  approach  to  estimating  Ada  software  costs.  A 
detailed  description  of  the  test  case  study,  results,  and  cxsnclusions 
is  included  in  this  r^xjrt. 

1.3  REPORT  OUTLINE 

The  organization  of  this  r^xjrt  corresponds  to  the  defined  tasks 
outlined  in  the  study  objective: 


Description  of  the  Models  -  Section  2 

This  section  provides  ein  overview  of  the  philoscphy  b^iind 
each  model  with  regard  to  Ada  versus  other  high  order 
languages.  Each  model  is  described  with  regard  to  several 
potential  cost  drivers  and  issues  that  are  believed  to  have  a 
different  ijtpach  on  Ada  versus  non-Ada  projects.  A  matrix 
that  demonstrates  the  language  considerations  of  current 
models  is  included  in  this  section. 
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Section  3 


Ifiis  section  provides  an  overview  of  the  Ada  projects 
targeted  in  the  test  case  stucty.  The  methodology  used  to 
collect  and  validate  project  data,  derive  the  model  input, 
and  subsequently  interpret  the  results  of  the  models 
applications  is  presented. 

•  Analysis  nf  Arfa  Project  Data  -  Section  4 

This  section  describes  the  sensitivity  analysis  performed  to 
validate  cost  drivers  previously  identified  and  uncover  new 
cost  drivers  within  an  application  dcmain. 

■  Conclusion  -  Section  5 

This  section  sunmarizes  findings  fron  each  of  the  defined 
tasks  described  in  the  previous  sections.  An  approach  to 
estimating  Ada  software  develc^xnent  costs  is  reccmmended  that 
considers  the  differences  in  the  model  philosophies,  cost 
drivers,  and  performance  in  view  of  the  dcnain  of  Ada 
projects  targeted  in  the  test  case  stu^.  Lessons  learned  on 
data  aquisition  and  data  aneilysis  are  also  included  in  this 
section. 

1.4  TEFMINOIDGY 

The  following  definitions  pertain  to  the  terminology  used  within 
this  study. 

Ada;  A  programming  language  that  was  designed  and  developed  by  the 
United  States  D^jartment  of  Defense  (DoD)  to  establish  one  ocnmcn 
programming  language  for  edl  its  applications  and  inocnpatible  dialects 
used  by  its  programmers. 

Consistency:  The  ocnparisOT  of  the  project's  actual  effort  and 
schedule  duration  to  the  model's  estimated  effort  and  schedule  after  a 
conputed  mean  value  heis  been  applied  to  each  model  estimate. 

Incremental  Develoanenti  A  software  development  paradigm  characterized 
by  implementing  and  testing  one  peirt  of  a  system,  then  iitplementing  and 
testing  another,  one  by  one,  until  the  system  is  ccnplete. 

Instantiation:  The  process  of  r^resenting  an  abstraction  by  a 
concrete  example.  In  Ada,  the  instantiation  of  a  generic  creates  a  new 
subprogram  or  package  that  can  be  used. 

Model:  The  underlying  mathematical  equation  or  integrated  set  of 
equations.  For  exanple,  COOCM3  is  a  single  equation  cost  model;  roiCE 
S  is  an  integrated  system  of  equations  and  operations. 
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rtomal ization:  The  process  of  ocaiforming  or  reducing  to  a  standard 

answer. 

Ctoiect  Oriented  Design;  A  partiticxiing  of  a  software  system  according 
to  classes  or  objects,  not  functions. 

Paclcaae;  An  inpleauentation  of  a  given  model.  For  example,  SEOOMD  and 
COfflMCDL  are  both  implementations  of  the  C300CMD  model. 

Pilot  Develocment!  An  initial  program  limited  in  either  scope  or 
functionality  of  the  final  system  to  test  the  feasibility  of 
inplementing  a  new  capability. 

Program  T.-ihrarv;  An  organized  r^jository  of  ramsable  code. 

Prototyping;  The  process  of  developing  an  early  model  that  r^resents 
the  actual  product. 

Relative  Error;  Ihe  comparison  of  estimated  effort  or  schedule 

duration  to  the  actuaLL  effort  or  schedule  losing  the  following  measure; 

relative  error  =  (estimated  effort  -  actaial  effort) /actual  effort 

Spiral  Develooment;  A  software  development  peiradigm  characterized  by 
successively  refined  understandings  of  the  problem  and  successively 
refined  prototypes  of  a  software  solution  to  the  problem. 

Structured  Analysis;  The  activity  of  deriving  a  functional  model  of 
the  requirements  for  a  system. 

Wateirfall  Development;  A  software  development  paradigm  characterized 
by  a  progression  throu^  a  series  of  steps  (Requirements,  Design, 
coding.  Testing) ,  each  followed  by  some  form  of  verificatiOT  (Software 
Requirements  Review,  Critical  Design  Review,  etc.). 
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2.0  DESCRIFnCN  OF  THE  MXELS 


2.1  ATA  OVERVIEW 

Models  that  are  included  in  this  study  were  selected  based  i^n 
their  availability  to  either  the  AFCSTC,  USACEAC,  or  IITRI.  Model 
vendors  were  not  solicited  for  their  participation.  In  addition, 
applicability  of  the  models  to  the  projects  targeted  in  the  test  ceise 
study  was  a  further  consideraticai.  For  exaitple,  Estimacs  is  a  model 
used  for  estimating  the  cost  of  business-type  applications.  Projects 
targeted  in  this  study  are  primarily  ccmmand  and  control  and 
tools/environment  applications.  Althou^  Estimacs  was  available  to  the 
AFCSTC,  it  was  not  included  in  this  study  because  of  its  applicability. 

Software  cost  estimation  models  were  reviewed  with  regard  to 
potential  cost  drivers  and  issues  that  have  a  different  impact  on  Ada 
versus  non-Ada  projects.  The  review  focused  on  two  Ada-specific  models 
used  to  estimate  development  costs  for  projects  in  vhich  Ada  is  the 
primary  software  language,  and  four  non-Ada-specific  models  apprcpriate 
for  application  to  Ada  projects  and  other  HOLs: 


Ada-Specific  Models 

Non-Ada-Specific  Models 

1.  Ada  COOCMD  (IOC) 

1. 

PRICE  S  (188230) 

2.  SoftCost-Ada  (1.3) 

2. 

SYSTEM-3  (1.03) 

3. 

SPQIV20  (1.2) 

4. 

SASET  (1.5) 

The  package  version  of  each  model  reviewed  is  indicated  in 
peirenthesis  beside  the  model  name.  The  OOSTMODL  package.  Version  5.0, 
vhich  implements  the  complete  Initial  Operational  Capability  (IOC) 
version  of  the  Ada  OXX^IO  model,  was  used  for  the  purposes  of  this 
study.  COSTMODL  (5.0)  implements  the  Ada  OOOCMD  model  introduced  by 
Dr.  Boehm  at  the  1987  COOCMD  User’s  Group  Conference.  It  does  not 
include  the  enhancements  to  the  Ada  model  that  Dr.  Boehm  introduced  at 
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the  1988  GOOCMD  Users'  Grcfup  Conference.  (The  1988  enhancements  are 
discussed  in  ^:ipendix  F) . 

Ihe  purpose  of  the  review  was  to  produce  a  summary  of  the  Ada 
technology  considerations  for  each  model.  The  results  of  the  review 
are  presented  in  the  following  sections.  Section  2.1  will  focus  on 
each  model  viiile  section  2.2  focuses  on  each  potential  cost  driver  and 
issue. 

Model  overviews  presented  in  this  section  include  the  following 
infornati(^  (when  available) : 

•  Philosophy  with  regard  to  Ada 

•  Recommended  responses  to  certain  model  inputs  for  Ada 

•  Ada  projects  used  to  develop  and  validate  the  model 

■  Clientele  vho  use  the  model  for  Ada  projects. 

General  information  regarding  hardware,  availability,  and  scope  of 
coverage  is  also  provided  for  each  model  in  the  report  appendices. 

2.1.1  Ada  OOOOM9 

The  Ada  OXXMD  model  is  a  tailored  version  of  the  standard  OCXXMD 
model  developed  by  Dr.  Barry  W.  Boehm  of  TRW's  Software  Information 
Systems  Division  and  described  in  full  detail  in  Boehm's  book.  Software 
Engineering  Economics  [B0EH81]. 

Although  the  existing  COCOMO  model  has  done  reasonably  well  in 
stpporting  Ada  cost  estimates,  experience  to  date  at  TFW  has  indicated 
that  a  version  of  OOOCMO  specifically  tailored  to  Ada  would  probably  do 
better.  One  of  the  reasons  for  developing  a  new  model  was  to 
incorporate  the  TPW  Ada  Process  Model  vAiich  assumes  a  phased 
increment£LL  develcproent  process  instead  of  a  waterfall  development 
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paradigm.  Biased  incremental  development  can  also  be  adapted  to  other 
ptograaining  languages  [B0EH87]. 

Barry  Boehm  and  Walker  Royce  led  develcpnent  of  the  Ada  version  of 
OOOCMD  in  a  process  that  began  in  1985-86  with  an  initicd  version  of 
the  Ada  Process  Model  and  an  initial  set  of  hypotheses  viiich  dealt  with 
the  effect  of  Ada  on  standard  COOCMO  cost  drivers.  In  ecirly  1987,  a 
two-phase  Delphi  process,  involving  10  TRW  personnel  experienced  in 
Ada,  softW2une  cost  estimation,  ard  large  software  projects,  was 
enployed  to  refine  the  functioral  form  of  Initiad  Operations  Capability 
(IOC)  Ada  COOQMO,  and  to  determine  an  initial  set  of  revised  cost 
driver  multipliers  and  exponents  (BOEH87]. 

In  mid-1987,  this  initicil  model  weis  ceLlibrated,  using  two 
completed  TFW  Ada  projects  vAiich  had  been  developed  using  full  DoD 
software  acquisition  standards.  Subsequently,  the  model  development 
process  and  product  were  subjected  to  an  independent  TRW  review, 
resulting  in  some  further  refinements.  The  resulting  IOC  Ada  COCCMD 
Model  was  presented  at  the  Third  OOCCM)  User's  Group  Meeting,  at  the 
Softwcure  Engineering  Institute,  in  November  1987  [BOEH87]. 

Differences  between  the  standard  COOCMO  and  Ada  OOOOMO  reflect  a 
E*u.los(^y  that  Ada  eiperienoes  are  like  that  of  any  other  new  language 
if  the  development  process  is  implemented  using  the  techniques  and 
milestones  of  most  previous  software  development  projects.  Nominal 
recalibration  of  required  reliability,  coarplexity,  and  language 
experience  is  the  impact  of  the  Ada  language.  The  more  significant 
impact  is  the  development  process  —  which  is  reoriented  to  capitalize 
on  Ada  strencfths  and  to  reinforce  more  efficient  software  production 
methods.  These  effects  include  modified  eiponential  scaling  equations 
for  estimating  nominal  development  effort,  schedule,  and  maintenance 
effort.  Phase  distributions  for  effort  and  schedule  are  revised. 

The  overall  functional  form,  most  of  the  effort  multipliers, 
adaptation  equations,  activity  distribution  tables,  and  the  use  of 
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Annual  Change  Traffic  for  maintenance  estimation  remain  the  same  in 
both  standard  and  Ada  CtXXMD.  The  IOC  Ada  OOOCMO  model  includes  scB»e 
general  improvements  which  also  apply  to  standard  OOOOMD  [BOEH87] : 


There  is  a  wider  range  of  ratings  emd  effects  due  to 
software  tools  (TOOL)  and  turnaround  time  (TURN) . 

Virtual  machine  volatility  (VIRT)  are  split  into  host 
machine  (VMVH)  and  target  machine  (VMVT)  effects. 

The  added  cost  due  to  schedule  stretchout  is  eliminated. 

Cost  drivers  aire  added  to  cover  the  effect  of  classified 
projects  (SECCJ)  cind  development  for  software  reusability 
(RUSE) . 

A  capability  is  added  to  estimate  the  costs  and  schedules 
involved  in  using  a  phased  incremental  development 
process. 


Other  changes  relate  to  the  effects  that  aire  specific  to  Ada  and  the 
TRW  Ada  Process  Model  [  BOEH87  ] : 


Cost  cind  schedule  effects  specific  to  Ada  cortprise  the 
following  changes  to  standcird  OOOOMO: 


1.  Multiplier  penalties  for  hi^er  levels  of  required 
reliability  (RELY)  and  ccitplexity  (CPLX)  are 
reduced. 

2.  There  is  a  wider  range  of  ratings  and  effects  due 
to  programming  language  experience  (LEXP) . 

3.  There  are  new  Ada-oriented  instructions  for 
counting  lines  of  code  and  reusing  software  in  Ada. 


Effects  of  using  the  TRW  Ada  Process  Mcxdel,  which 
reorients  portions  of  the  softwEune  develcpnent  process, 
result  in  the  following  cost  and  schedule  effects: 


1.  Exponential  scaling  equations  for  nominal 

development  effort,  development  schedule,  and 
ncmincil  maintenance  effort  are  revised. 
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2.  Hie  range  of  effects  of  modem  programming 
practices  (MDDP) ,  aneilyst  capability  (ACAP) ,  and 
programmer  capability  (PCAF)  have  been  extended. 

3.  Ftiase  distributions  of  effort  and  schedule  are 
revised. 

The  general  inprovements  to  OOCCMD  and  the  language-specific 
modifications  apply  to  the  stcindard  waterfaill  model.  They  cilso  apply 
to  a  single  increment  use  of  the  Ada  Process  Model  along  with  the 
changes  due  to  the  Ada  Process  Model .  For  a  phased  incremental 
development,  changes  due  to  the  Ada  Process  Model  apply  [EDEH88]. 

Althou^  Ada  OOOCMO  is  not  available  in  automated  form  directly 
frxan  Boehm  or  TRW,  there  are  several  automated  iirplementations 
commercially  available  from  other  organizations.  These  include  NASA 
Johnson  Space  Center's  OOSTMODL;  OOSTAR  developed  by  Softstar  Systems; 
GEOOMO  developed  by  GEC  Software;  and  an  IOC  Ada  OOOOMO  version 
implemented  by  the  Ballistic  Missile  Office  (EMO-ACS) .  Appendix  A 
lists  specific  points  of  contact  for  Ada  COCOMO  implementations.  When 
inguiring  about  a  package  implementation  of  Ada  COCOMO,  it  is  important 
to  note  vhich  version  of  the  model  the  package  implements,  (i.e. ,  1987 
IOC  version,  1988,  etc.)  and  whether  the  package  implements  the 
incremental  development  model. 

A  detailed  description  on  hew  to  generate  an  estimate  with  the  Ada 
COCOMO  model  is  provided  in  Appendix  F.  Note  that  current  estimating 
equatiens  only  cpply  to  Intermediate  OOCCNO,  eobedded-mode  software 
projects  (see  pages  78  -  83,  Software  Engineering  Economics  [B0EH81]). 
Equations  have  not  been  developed  for  organic  or  semidetached  modes  of 
Intermediate  COCOMO.  Basic  COCOMO  effort  eind  schedule  equations  are 
also  not  included  in  the  IOC  version  of  Ada  COCOMO. 

Because  Ada  COCOMO  is  an  initicil  eperating  capability  calibrated 
to  only  two  ccnpleted  Ada  projects,  use  of  the  model  is  not  widespread. 
TRW  uses  the  model  in  parallel  with  stcindard  COCOMO  and  other  Ada  cost 
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itodels  to  increase  perspective  on  Ada  cost  estimates.  TFW  will 
continue  Ada  project  data  collection  for  further  refinement  and 
calibration  of  Ada  OOOCMO  [B0EH87]. 

2.1.2  SoftCost-Ada 

SoftCost-Ada  is  a  derivative  of  RCI's  SoftCost-R  model,  v^iich 
generates  schedule  and  cost  estimates  for  systems  iuplemented  in 
conventional  programming  languages.  When  SoftCost-R  was  initially  run 
on  Ada  project  data,  the  results  were  inconsistent.  Attenpts  were  made 
to  modify  the  model  for  Ada  projects,  but  the  language  mar3ced  such  a 
shift  in  the  philosophy  of  programming  languages  that  new  equations 
were  developed  specifically  for  it.  Four  cost  drivers  were  identified 
for  costing  an  Ada  project  that  were  not  addressed  in  SoftCost-R 
[RCI88]; 

•  Ada  Usage  factor:  the  percentage  of  code  ejgjected  to 
be  written  in  Ada. 

•  Degree  of  Real-Time:  the  relative  complexity  of  the 
tasking  to  be  performed  in  the  system. 

■  Degree  of  Reuse:  the  percentage  of  new  code  expected 
to  be  packaged  for  reuse,  both  internal  and  external 
to  the  project. 

•  Resource  Availability:  the  availability  of  resources, 
such  as  staff,  tools,  equipment,  etc. ,  within  the 
developer's  organization. 


Cost  drivers  addressed  in  SoftCost-R  which  have  a  significant  inpact  on 
Ada  developments  include  [RCI88]: 

■  Degree  of  Standardization 

•  Use  of  Modem  Software  Methods 

•  Use  of  Software  Tools/Environments 

•  Software  Tools/Environments  Stability. 
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Data  used  to  develop  the  SoftCost-Ada  model  consisted  of 
approximately  30  software  projects  developed  by  12  different 
organizations  within  five  aerospace  firms  during  the  period  spanning 
1982  throu^  1987.  Projects  by  application  area  included  autonation, 
canmand  and  control,  conmercial  avionics,  embedded,  environment/tools, 
teleccOTTunications,  and  simulator  softwcure.  Today,  RCI's  database 
consists  of  more  than  90  projects,  vAiich  r^resent  more  than  25  million 
lines  of  Ada  source  code  [RCI88].  Clientele  v«4io  use  the  model  for  Ada 
projects  include  Rockwell,  Shell  Oil,  Singer  Link,  Westinghouse,  AAI, 
Contel,  Nippcxi  Telegraph  and  Tel^jhone  (Japan) ,  U.S.  Air  Force,  END 
(Canada) ,  Ihilips  (Sweden) ,  and  Ferranti  (England) . 

2.1.3  FRIGE  S 


Note;  The  following  description  is  condensed  from  "Ada  Estimating- 

-  A  PRICE  S  Profile”  pr^>ared  by  Dr.  Robert  E.  Park,  PRICE 

Systems  [Park89] 

PRICE  S  af^roaches  estimating  for  Ada  in  the  same  way  that  it 
approaches  estimating  for  other  develc^xnent  environments.  Costs  and 
schedules  are  driven  more  strongly  by  product  chcuacteristics  and 
developer  practices  than  by  development  languages.  However,  PRICE  S 
does  recognize  that  different  source  languages  have  different  stpport 
environments  and  different  productivity  characteristics.  Seme 
languages,  like  Ada,  even  iitply  different  iianagement  methodologies. 

When  Ada  is  specified,  PRICE  S  does  three  things: 

-  It  uses  relationships  appropriate  to  Ada  to  translate 
the  number  of  source  lines  of  code  into  the  model's 
internal  measure  for  product  size. 

-  It  uses  cin  Ada-specific  technology  submodel  to 
estimate  the  trends  in  productivity  improvement  that 
occur  as  a  function  of  time. 

-  It  distributes  the  activities  of  designers, 
programmers,  systems  engineers,  and  program  managers 
more  toward  the  early  phases  than  it  does  for  other 
development  environments. 


otherwise,  all  parameters  in  PRICE  S  are  used  and  evciluated  just 
cis  in  either  software  estimates.  Estimators  cue  not  asked  to  chcinge 
tools  or  procedures  in  order  to  estimate  Ada  projects. 

When  configuring  PRICE  S  for  Ada  estimating,  users  pay  special 
attention  to  parameters  that  take  on  values  different  fron  those  used 
to  describe  their  reference  baselines.  For  exairple,  if  the  reference 
projects  contain  no  Ada  software,  then  the  difference  expected  with 
must  be  spelled  out.  PRICE  S  recemnends  that  the  estimator  examine 
the  project  being  estimated  may  differ  from  the  reference  baseline  w 
regard  to  the  following  factors: 

Resource  attributes: 

•  productivity  factors 

•  Schedule  distribution 

•  Cost  element  distribution 

•  Labor  rates  (if  different  personnel  are  used 
and  monetary  costs  are  desired) 

personnel  attributes: 

•  Team  experience 

•  Team  sJd.ll 

Product  familiarity: 

■  Extent  of  experience  with  similar  software 

•  Extent  of  experience  with  software  of  similar  size 

Software  tools 

New  programming  language 

New  corputer  hardware 

Unusual  levels  of  virtual  machine  experience 

Unusual  levels  of  virtual  machine  volatility 

Unusual  requirements  to  produce  reusable  code  (generics,  for 
example) 

Intrcduction  of  new  practices  for  software  design  and 
programming 


2-8 


III 


These  are  the  cost  cxsiimon  ways  in  which  Ada  projects  can  differ  from 
their  predecessors.  ERICE  S  parameters  permit  the  responses  for  each 
assessment  to  be  incorporated  into  an  estimate. 

Estimating  with  FKECE  S  is  easiest  i^hen  ccrpleted  Ada  projects  are 
available  for  Ccdibration.  Here,  organizationcil  productivities, 
scheduling  practices,  and  cost  distributions  can  be  meeisured.  These 
measurements  can  be  used  to  configure  the  model  to  fit  demonstrated  Ada 
performance.  Differences  between  new  projects  and  calibrated 
references  then  become  smaller  and  easier  to  judge  than  when 
ccnparisons  must  be  made  against  non-Ada  baselines. 

In  cill  cases,  FKECE  S  philosophy  recognizes  that  software 
developers  differ  and  that  develcpment  environments  are  dynamic.  The 
philosc^y  stresses  that  the  most  reliable  basis  for  an  estimate  is 
that  of  extrapolation  frcm  demonstrated  performance.  As  tools  and 
management  methods  evolve  to  talce  advantage  of  Ada,  users  of  FKECE  S 
will  find  some  parameters  taking  on  different  values,  but  the 
procedures  for  estimating  will  not  need  to  change. 


2.1.4  SASET 

In  1986  while  under  contract  to  the  Naval  Center  for  Cost 
Ancilysis,  Martin  Marietta  conducted  a  study  of  existing  software 
estimating  and  sizing  models.  In  its  findings,  Martin  Marietta 
determined  that  there  was  a  need  for  a  forward-chaining,  rule-betsed 
expert  system  utilizing  a  hierarchically  structured  kncwledge  database 
to  provide  functional  software  sizing  values,  c^imal  development 
schedule,  and  associated  manloading  outputs.  Martin  Marietta  develc:ped 
the  ccsxputerized  model  Software  Architecture  Sizing,  and  Estimating 
Tool  (SASET)  as  a  prototype  in  1987  that  would  satisfy  these 
requirements  [CHEA89]. 

SASET  is  a  rule-based  ejqort  system  that  utilizes  a  "three-tiered" 
approach  for  system  identification.  Tier  1  -  System  Environment- 
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requests  information  regarding  the  class  of  software,  programming 
language,  developnsent  schedule,  development  locations,  and  other 
factors  vhich  descrihe  the  development  environment.  Oul^jut  from  this 
tier  are  factors  used  in  subsequent  processing  to  derive  budget  and 
schedule  estimates  [MARTSS]. 

Tier  2  -  Software  Functionality  -  autcmates  the  process  of  sizing 
by  analogy.  The  user  indicates,  throu^  a  process  of  functional 
deccmposition,  those  software  functions  to  be  contained  in  the 
developed  software  system.  A  ncmincil  SIix:  estimate  obtained  from  the 
SASET  database  for  each  of  the  individual  functions  is  adjusted  based 
vjpon  the  responses  provided  for  each  software  element.  The 
information; 

o  Perceived  corplexity, 

o  Degree  of  new  develc^ment, 

o  Develcpnent  lariguage  (HOL  or  Assembly) ,  arxi 
o  Hosting  CHJ 


is  used  to  adjust  the  function's  base  sizing  estimate  and  derive  a 
lines  of  code  value.  The  tier  2  sizing  estimate  is  in  equivalent 
FORTRAN  lines  of  code  excluding  ccmroents.  The  user  may  also  enter 
sizing  informatin  directly  rather  than  identify  specific  software 
functions  [MARTSS]. 

Tier  3  -  System  Conplexity  -  requires  the  user  to  rate  20 
ocnplexity  issues  on  a  scale  of  l  (very  ocnplex)  to  4  (simple) .  A 
wei^ting  factor  is  assigned  to  each  response.  The  product  of  the 
wei^ting  factors  represents  the  impact  to  the  software  develc^ment 
budget  and  schedule  [MARTSS]. 

A  fourth  tier  is  used  to  estimate  manloading  over  the  entire 
maintenance  life  cycle. 
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Martin  Marietta  software  develcpment  data  consisting  of  more  than 
300  <xnpleted  projects  and  scxne  selected  Navy  data  were  xjsed  to  develop 
the  SASET  model  [CHEAS9].  SASET  is  meant  to  be  used  to  estimate 
development  in  Assembly,  Ada,  or  any  HOL.  However,  the  model  does  not 
differentiate  between  Ada  and  non-Ada  development  in  its  input 
responses  exo^jt  in  the  following  instances  [MARTSS]: 


o  Ihe  tier  3  input  for  cxnplexity  of  the  programming  language 
has  a  direct  effect  on  the  produo±ivity  of  the  software 
programmers.  The  selections  that  SASET  provides  penalize  Ada 
projects  for  their  ''cxmplex  rigidly  structured  constructs 
(training  usually  required)." 

o  The  tier  1  input  for  primary  software  language  has  an  effect 
on  the  software  develc^itent  effort-schedule  due  to  the 
robustness  of  the  software  language  and  the  type/capability 
of  the  system  being  developed.  The  tier  1  model  input 
allocates  software  languages  into  the  following  four  classes; 

1.  Ada 

2.  LISP,  reoIDG,  C,  Assembly 

3.  Traditional  HOLs:  FORTRAN,  Pascal,  JOVIAL 

4.  User  languages.  Simple  Non-structured  HOIs  and  Test 
Sequenc:e  languages. 

SASET  was  initially  developed  as  a  manual  model  to  MIL-STD-483, 
490,  1679,  and  1521A  requirements.  It  has  been  recommended  that  the 
csonputerized  model  be  v;pdated  to  also  reflect  a  DoD-STD-2167A 
develcproent.  In  doing  so,  the  following  Ada  items  would  be  addressed 
[CHEA89]: 


o  Modem  Software  Engineering  Practices 
o  Reusability 
o  Ada  language  Features 

o  APSE  Tools  (Ada  Programming  Support  Environment) . 


Clientele  who  use  the  SASET  model  for  Ada  projects  are  the  Naval 
Center  for  Costs  Analysis  and  the  Air  Force  Cc5st  Center. 
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2.1.5  SKm/20 


Software  Productivity,  Quality,  &  Reliability/20  (SPQIV20)  is  a 
cost  and  quality  estimation  mcxiel  for  planning  software  development  and 
maintenance  activities.  Ihe  model  was  developed  by  T.  Capers  Jones  of 
Software  Productivity  Research,  Inc.  (SIR)  based  on  historical  data 
frcm  more  than  3,000  software  projects  and  more  than  200  organizations 
[SERBS] .  The  database  currently  includes  approximately  50  Ada  projects. 

SPOEVZO  views  Ada  like  other  HOLs.  Irputs  are  d^jendent  upon  the 
particular  application.  SPC2EV'20  assigns  a  default  value  of  4.5  to  Ada 
for  the  numeric  level  of  the  language  relative  to  assembler  which  the 
user  is  able  to  change. 

SPQEV'20  differentiates  between  several  different  develcpment 
peuadigms  including  waterfadl  method,  incremental  build,  and  spiral 
develcpment.  The  significant  paradigm  enplcyed  by  SPQEV20  is  that  of 
pattern  matching.  The  mathematical  modeling  Jones  uses  for  the  tool  is 
based  on  "circa"  3,500  projects  including  military  real-time  and 
embeckJed  systems,  and  MIS  projects.  The  algorithms  that  drive  the 
estimates  are  derived  from  this  database  which  is  constantly  being 
updated  [SFR88]. 

2.1.6  SYSTEM-3 


SYSTEM-3  is  a  cost  estimation  product  developed  in  1985  by 
Conputer  Etoonondcs,  Inc.  (CEI) .  SYSTEM-3  and  its  predecessors,  JS-1 
(1979)  and  JS-2  (1984),  were  develcped  by  Dr.  Randall  W.  Jensen,  based 
ipon  the  research  of  cost  estimation  by  Norden  (1970) ,  Putnam  (1976) , 
Doty  (1977),  Jensen  (1979),  and  Boehm  (1981)  [inR87].  SYSTEM-3 

reflects  a  philcascphy  that  Ada  experiences  are  like  that  of  any  ether 
new  language.  While  the  model  takes  into  accxiunt  language  and  tool 
maturity,  modem  software  engineering  practices,  the  leeuning  curve  and 
reusability  irpacts,  these  issues  are  not  treated  as  uniejue  phencmena 
to  the  Ada  language  [CEI88]. 
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The  same  ii^juts  are  required  for  Ada  and  non-Ada  projec±s.  There 
are,  however,  recxatimended  inputs  for  several  peirameters  that  describe 
Ada  development.  The  recommended  minimum  (MIN) ,  nominal  (NCM) ,  and 
maximum  (MAX)  values  for  Ada,  shewn  in  Table  2-1,  define  where  the  user 
is  with  regard  to  Ada  experience  [GAL86] . 

TABIE  2-1 


RBCCM1ENIXS  SY5I1M-3  INIVIS  FOR  ADA  EEVEIDIHEMr 


Input  Kcameter 

IBI 

NCM 

MAX 

1. 

Ada  Language  Complexity 

3 

3 

3 

2. 

Ada  language  Ebqarience 

3 

3 

4 

3. 

Ada  Virtual  Machine 
Ejqjerience 

3 

3 

4 

4. 

Ada  Virtual  Machine 
Volatility 

1 

1 

2.5 

5. 

Ada  Use  of  Modem 

Development  Practices 

5.5 

7.5 

8 

6. 

Ada  Virtual  Machine 
Complexity 

2 

2 

2 

7. 

Ada  Automated  Tool  Usage 

5 

6 

7.5 

8. 

Ada  Turn  Around  Time  * 

0.5 

0.5 

0.3 

None  of  the  recently  completed  Ada  projects  targeted  in  this  study 
(see  section  3.2,  Overview  of  Ada  Projects),  indicated  response 
times  greater  than  six-minutes  (rating  was  set  to  0.1) . 


Clientele  who  use  the  model  for  Ada  projects  include  the  USAF  AFSC 
Divisions  (ASD,  BSD,  BD,  AD,  EMD) ,  and  D^artment  of  Transportation 
(DOT)  Canada. 
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2.2 


Conmercial  software  cost  models  were  reviewed  in  light  of  Ada- 
specific  cost  drivers  and  issues.  A  primary  resource  used  to  identify 
these  issues  was  a  r^xart  pr^ared  for  the  Electronic  Systems  Division 
(ESD) ,  Air  Force  System  Command  entitled  A  Plan  for  Collecting  Ada 
Software  Develocment  Cost.  Schedule,  and  Environment  Data  (CR-0134/1) , 
dated  2  ^ril  1987  [TECOS?].  This  study  identified  several  potential 
cost  drivers  and  issues  that  have  different  impact  on  Ada  projects 
versus  non-Ada  projects.  They  are: 

1.  Time  spent  in  design 

2.  Allocation  of  costs  to  phases  in  the  life-cycle 

3.  Modem  software  engirkeering  practices 

4.  Lecuning  curve  for  the  Ada  language  and  tools 

5.  Reusability  requirements 

6.  Use  of  certain  Ada  language  features 

7.  Ada  Programming  Sipport  Environment  (APSE) 
productivity  tools 

8.  Language  and  tool  maturity 

9.  Influence  of  personnel  assigned  to  Ada  projects  and 
the  fact  that  the  projects  are  closely  monitored 
(Hawthorne  Effect) . 

The  Ada  issues  were  identified  from  literature  and  a  series  of 
interviews  with  software  modelers,  Ada  software  engineering  experts, 
and  Ada  project  managers  [TEC087].  For  each  model,  Ada  issues  were 
mapped  to  corresponding  irput  pcurameters,  if  present,  cind  categorized 
as  one  of  the  following: 
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1.  Pecalibrated  for  develcproent  process:  New  cost  driver (s)  are 
introduced,  exponential  scaling  equations  are  revised,  or 
cost  driver  ratings  are  revised  for  a  develcpment  process 
vAiich  is  reoriented  for  Ma. 

2.  Recalibrated  for  Ada:  New  cost  driver (s)  are  introduced, 
exponential  scaling  equations  are  revised,  or  cost  driver 
ratings  are  revised  due  to  features  of  the  Ada  programming 
language  regardless  of  the  develcpnent  process. 

3.  Cost  driver  present  tut  not  Ada-unique:  Ihe  issue  is 

reflected  by  the  response  of  the  user  to  the  model's  input 
parameter  (s) .  Cost  driver  ratings  are  not  recalibrated  for 
Ada.  Ihe  specified  issue  is  not  treated  cis  Ada-unique 
phenomena. 

4.  Externally  recalibrated  by  user:  Completed  Ada  projects  by 
the  developing  organization  can  be  used  to  configure  the 
model  to  fit  demonstrated  Ada  performance  with  regard  to  the 
specified  Ada  issue. 

5.  No  influence.  The  specified  issue  has  no  iirpact  on  software 
development  cost. 


Table  2-2  provides  a  summary  of  the  language  considerations  of 
each  model  applied  in  the  test  case  study.  Each  Ada  issue  is  further 
described  in  the  sections  that  follow. 

2.2.1  Ehase  Distribution  of  Effort 

Phase  distribution  of  effort  entails  the  allocation  of  time 
throu<^out  the  requirements,  design,  inplementation  and  testing  phases 
of  the  development  cycle.  Experts  cited  in  the  ESD  report  said  that 
the  effort  distribution  of  an  Ada  project  may  differ  from  projects 
using  other  languages  because  of  the  enphasis  placed  on  the  early 
stages  (requirements  and  design)  of  the  life-cycle  [TE0087].  One 
factor  that  will  affect  the  effort  distribution  is  the  newness  of  the 
language.  Because  Ada  is  a  relatively  new  language  and  contains  unique 
features  not  found  in  other  languages  (generics,  tasks,  exception 
handlers) ,  the  early  phases  will  take  more  time  in  order  to  correctly 
learn  and  use  the  language  properly.  Also  effecting  the  phase 
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LANGUAGE  CONSIDERATIONS  OF  CURRENT  SOFTWARE  COST  MODELS 
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Development  of  Reusable  Components/Use  of  Reusable  Components 


distritution  is  the  newness  and  cxrplexity  of  the  development  process 
most  oonmonly  used  on  Ada  projects,  object  oriented  design.  This  type 
of  methodology  is  kncwn  to  force  interd^jendencies  between  modules  in 
the  requirements  and  design  phases  of  the  development  cycle  [TE0087]. 
Ada  provides  package  specifications  to  set  these  interd^jendencies 
vhich  beccroe  the  bottom  layers  of  the  software.  A  project  can  incur  a 
serious  peneQ.ty  if  these  specifications  are  changed  in  later  phases  due 
to  the  effects  a  change  will  have  on  other  modules  vhich  interact  with 
the  changed  specification. 

Examining  the  models  in  li^t  of  this  issue  shews  two  different 
views  of  the  effect  of  Ada  on  the  phase  distribution.  One  view  is  that 
the  effort  distribution  should  not  change  because  of  the  Ada  Icinguage, 
but  should  change  to  fit  the  observed  performance  of  the  develqpers. 
This  view  is  found  primarily  in  the  non-Ada  specific  models,  namely 
SASET,  SPQIV20,  and  SYSTEM-3,  and  EMCE  S.  These  models  do  not  provide 
an  internal  effort  equation  specific  to  Ada,  but  they  do  supply  a 
nominal  effort  equation  that  can  be  recalibrated  extemcilly  by  the 
user.  The  other  view  is  that  the  phase  distribution  of  effort  is 
different  for  Ada  and  is  reflected  in  the  model's  underlying  equations. 
This  view  is  present  in  the  remaining  two  cost  models,  Ada  COOOMO  and 
SoftCost-Ada. 

Ada  G0GCN3.  The  TFW  Ada  Process  Model  reorients  portions  of  the 
software  develc^ment  process  to  capitalize  on  Ada  strengths  and 
reinforce  more  efficient  software  production  methods  [B0EH88].  The 
Process  model,  vhich  can  be  adapted  to  non-Ada  projects,  is  inherent  to 
the  Ada  version  of  OOOCXIO.  If  the  Ada  Process  model  is  being 
implemented,  phase  distribution  of  effort  will  shift  from  integration 
and  test  to  design  phases.  Table  2-3  shews  the  schedule  and  effort 
distribution  by  phase  if  there  is  compliance  with  the  Ada  Process 
Model.  A  similar  breakout  of  effort  for  standard  aX3CM0  is  shewn  for 
conparison.  Partial  ccmplicince  with  the  Ada  Process  Model  will  e^ibit 
a  phase  distribution  in  between  that  shewn  for  standcurd  COCCMD  and  Ada 
CDOCMO. 
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TABLE  2-3 


aHPARISOJ  OF  ADA  OOOCMD  AND  STANDARD  OXJCMD  SCHEDULE 
AND  EFFORT  DISTRIBCTITON  BY  PHASE. 


Standard  OOOOMO:  [Large  Embedded] 


Phase  Distribution 

SSR-Pt® 

PCR-CER 

CCR-TRR 

TRR-FQR 

Effort 

18 

25 

26 

31 

Schedule 

36 

{  36  } 

28 

IOC  Ada  OOOCMD:  [Large  Embedded] 


Phase  Distribution 

SSR-PCR 

PCR-CER 

CCR-TRR 

TRR-PX^ 

Effort 

23 

29 

22 

26 

Schedule 

39 

25 

15 

21 

SoftOost-Ada.  SoftCost-Ada  *  s  allcxiation  of  costs  to  {biases  of  the 
life-cycle  differs  from  the  traditional  allocation  of  time  and  effort 
to  life-cycle  pAiases  [RCI88].  Table  2-4  provides  a  conparison  of 
SoftCost-R  versus  SoftCost-Ada  phase  distribution. 

The  effort  and  schedule  distribution  for  SoftCost-Ada  are  adjusted 
toward  the  40:20:40  allocation  if  the  Ada  development  process  follcws  a 
more  traditional  approach  used  for  other  HOLs. 
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TABLE  2-4 

SoftCost-R  VS.  SoftCost-ADA  PHASE  DISTRIBLTTION 


SoftCost-R 


Phase  Distribution 

SER-CCR 

CER-^IRR 

TRR-PQR 

Effort 

40 

20 

40 

Schedule 

45 

25 

30 

SoftCcJst-Ada 


Phase  Distribution 

SCR-CIR 

CER-TRR 

TRR-PQR 

Effort 

50 

15 

35 

Schedule 

50 

15 

35 

2.2.2  Modem  Software  Development  Practices 

The  software  development  process  encortpasses  both  the  wide  variety 
of  develcpment  paradigms:  waterfall  develcpnent,  incremental 
development,  spiral  development,  pilot  develqpment;  and  the  develcpnent 
practices:  object  oriented  design,  structured  analysis,  incremental 
develcpment,  prototyping,  program  libraries.  A  combination  of 
structured  analysis  and  object  oriented  design  (CX)D)  used  within  a 
waterfall  development  paradigm  are  develc^ment  processes  most 
frequently  associated  with  Ada  in  recent  efforts.  In  1985-1986,  TRW 
develcped  an  initial  version  of  the  Ada  Process  Model  that  reorients 
portions  of  the  software  develcpment  process  to  capitalize  on  Ada 
strengths  eind  to  reinforce  more  efficient  softwcire  production  methods. 
The  Ada  Process  Model  focusses  on  utilizing  the  prograraming-in-the- 
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large  features  of  Ada  (primarily,  corpilable  package  specifications)  to 
enhance  design  thorou^iness  and  eliminate  major  risk  items.  Ihe  Ada 
Process  Model  was  designed  to  eliminate  inefficiencies  of  most  previous 
projects  "brou^t  on  when  lau^ge  nuntoers  of  project  personnel  eire 

working  in  parallel  on  tasks  which  are  closely  intertwined, 

incotpletely  defined,  continually  changing,  and  not  well  prepared  for 
dcwnstreeun  integration  [B0EH87]''.  The  primary  features  of  the  Ada 
Process  Model  include  the  use  of: 

•  Snail  front  design  teams 

•  Planned  incremental  develcpnent 

•  Ccnpilable  package  specifications  by  PCR 

•  Continuous  integration  via  package  specifications 

•  Technical  walkthrou<^  proceeding  SSR  and  PDR. 

Tbe  Ada  Process  Model,  differs  substantially  frcsm  traditional 

development  practices  vrtiere  main  points  of  interest  are  the  Preliminary 
Design  Review  (PCR)  and  the  Criticcil  Design  Review  (CCR) .  Interviewees 
cited  in  the  BSD  Report  believed  that  as  time  progresses,  these 

conventional  milestones  may  no  longer  be  apprc^riate  for  Ada  softv^are 
development  [TEOD87].  Instead  they  believe  that  criteria  for  meeting 
milestones  might  coincide  with  the  following  points  of  interest: 

•  Noting  the  point  in  software  where  all  packages  and 
procedures  cure  named. 

•  Noting  vhen  the  type  statements  are  identified 

•  Determining  hew  many  objects  are  identified  at  all  of 
those  times. 
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Of  the  cost  models  reviewed  in  this  study,  only  SASET  and  SPQIV20 
did  not  take  this  issue  into  account.  SYSTEM-3,  FRIGE  S,  Ada  OOOOMD 
and  SoftCost-Ada  adjust  for  the  influence  of  modem  software 
engineering  practices  to  varying  degrees,  primarily  through  model  input 
parameters. 

SYSIHf-3.  SYSTEM-3  views  the  use  of  modem  develc^ment  practices 
as  an  influence  to  software  cost.  Ihese  influences,  however  are  not 
considered  unique  to  Ada.  Ihe  SYSTEM-3  input  parameter.  Modem 
Develcpjnent  Practices  (MODP) ,  rates  the  developer's  use  of  modem 
develc^ment  practices  throu^out  the  project.  Factored  into  the  rating 
for  MODP  are  the  develcpnent  team's  previous  ejqjerience  with  practices 
and  whether  the  practices  have  been  used  successfully  on  prior 
contracts. 

HUGE  S.  This  issue  is  accounted  for  in  the  productivity  factors 
and  schedule  multipliers  which  are  calibrated  to  include  the  effects  of 
current  software  engineering  practices  measured  from  completed 
programs.  Subsequent  application  of  these  factors  assumes  that 
conditions  present  in  the  calibrated  projects  continue  to  hold,  thus 
incorporating  the  effects  of  the  developers  experience  with  Modern 
Software  Engineering  Practices  [PAPK89] . 

Ada  CCXno.  Several  input  parameters  are  used  to  describe  the 
softweu:e  development  process: 
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Use  of  modem  programming  practices 
Experience  with  the  Ada  Process  Model 


•  Design  thoroughness  at  PER,  unit  package  specs  compiled, 
body  outlined 

•  Risks  eliminated  at  PER 

•  Requirements  volatility  during  develcpnent. 

Most  of  the  factors  rate  the  extent  of  compliance  with  the  Ada  Process 
Model  discussed  previously.  Use  of  modem  software  practices  and 
compliance  with  the  Ada  Process  Model  leads  to  an  overall  reduction  in 
project  effort. 

SoftC3ost-Ada.  SoftCost-Ada  reflects  the  modem  software 
engineering  practices  issue  in  three  of  its  input  parameters  and  its 
calibration  coefficients: 

•  Use  of  Modem  Software  ffethods 

•  Ada  Methodology  Experience 

•  Team  Capability. 

The  Use  of  Modem  Software  Methods  parameter  asks  the  user  to  enter  the 
type  of  software  method  being  used:  structured  programming,  OOD,  Ada 
packaging  methods,  etc.  Ihe  second  parameter,  Ada  Methodology 
Experience,  rates  the  develcper's  experience  with  the  chosen 
develcpnent  methodology.  If  experience  with  this  methodology  is  hi^, 
then  the  project  will  benefit.  Ihe  last  parameter.  Team  Capability, 
addresses  the  issue  of  the  type  of  teams  used  in  the  development. 
Participatory  and  interdisciplincUY  teams  are  a  significant  factor  in 
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the  software  being  developed  by  cxinsensus  rather  than  by  decnnee 
[RCI88]. 

2.2.3  Tpamina  QiTve 

The  leeiming  curve  is  driven  by  two  issues;  the  difficulty  of 
learning  the  language  and  the  amount  of  time  to  become  proficient  in 
it.  Ejq^erts  cited  in  the  ESD  r^xort  felt  that  the  learning  curve  in 
Ada  projects  may  have  more  cost  inpact  because  of  Ada's  novelty  and 
corplexity  [TEC3087].  Because  Ada  is  a  relatively  new  language  and  Ada 
Programming  Svpport  Environments  (APSEs)  are  still  evolving,  a  body  of 
trained  personnel  are  not  available  to  develop  new  Ada  systems.  Ada 
projects  will  not  be  able  to  benefit  from  lessons  learned  in  previous 
projects  which  in  turn  will  lengthen  the  develc^mvent  schedule.  In 
addition,  most  respondents  felt  that  Ada  requires  more  experience  than 
other  languages  before  personnel  can  become  proficient.  This  is 
because  of  the  unique  features  of  the  language  (generics,  tasks, 
overload  c^ierators,  exo^jtion  handlers)  tliat  are  not  present  in  other 
languages.  It  was  generally  agreed  that  it  will  ta)ce  longer  to 
appreciate  the  tradeoffs  in  determining  vhich  feature  of  the  language 
is  best  to  implement  a  particular  algorithm.  Althou^  the  time  to 
become  proficient  in  the  Ada  langueige  seemed  longer  than  other 
IcUTguages,  the  expaerts  also  believed  that  there  would  be  a  "fusion" 
pxjint  where  proficiency  in  the  language  wcwld  become  evident  and  the 
benefits  of  the  language  would  be  seen  in  better  design,  code,  and  more 
reusability  [TE0087]. 
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All  of  the  cost  models  studied  accounted  for  the  difficulty  of 
learning  a  language,  but  the  Ada-specific  models  were  the  only  models 
that  treated  the  Ada  learning  curve  differently  frcm  other  languages. 
For  each  model,  the  factor  for  learning  the  language  was  determined  by 
inputs  for  the  experience  of  the  project  personnel.  With  regards  to 
the  seocaYi  issue,  the  amount  of  time  to  beccme  proficient  in  the  Ada 
language,  only  Ada-specific  models  accounted  for  this  issue.  Iheir 
philosophies  are  discussed  below. 

Ada  G0aQM3.  Proficiency  in  Ada  versus  proficiency  in  other  HOIe 
can  be  assessed  by  cotrparing  the  factors  associated  with  the  language 
Ejqjerience  (lEXP)  irput  parameter  in  Table  2-5. 


TABI£  2-5 

OaiPARISa^  OF  STANDARD  OOOMD  AND  ADA  CXXXMO  LANGUAGE  EXPERIENCE  FACTORS 


Rating  level 

Rating  Scale 

Standard 

Ada 

VL 

<=  1  month 

1.14 

1.26 

L 

4  months 

1.07 

1.14 

NCM 

1  year 

1.00 

1.04 

H 

3  yecirs 

0.95 

0.95 

VH 

>=  6  years 

0.95 

0.86 
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As  the  chart  indicates,  larger  penalties  occur  becaiase  of  Ada  ignorance 
while  larger  benefits  occur  because  of  acquired  Ada  ejqjertise  [TFW88] . 

SofbOost-Ada.  A  significant  finding  of  RCI  researchers  was  that 
the  exponents  used  in  SoftCost-Ada  effort  equations  did  not  stabilize 
until  the  team  etssigned  to  the  project  hc»d  at  least  3-5  Ada  projects 
worth  of  Ada  e^qierience.  In  additicn,  e^q^erience  was  also  influenced 
by  the  following  factors:  analyst  capability,  applications  experience, 
environinent  experience,  and  language  and  methodology  experience.  Table 
2-6  d^icts  how  the  power  law  changes  eis  a  function  of  the  number  of 
Ada  projects  conpleted  and  the  other  contributing  factors  [RCI88]. 


TABLE  2-6 

POWER  lAW  AS  A  FlJNCnON  OF  THE  NUMBER  OF  ADA  PRCOECTS  OaOTETED 


Number  of  Conpleted  Ada  Projects 

PI 

P2 

0 

0.40 

1 

0.39 

2 

0.38 

3 

1.00 

0.37 

4 

0.95 

0.36 

5 

0.95 

0.35 

6 

0.95 

0.35 

PI  =  Effort  Exponent 
P2  =  Schedule  Exponent 


2-25 


2.2.4 


Reusability  includes  not  only  developing  reusable  software  but 
also  accounting  for  conponents  that  etre  reused  from  other  softwcure.  It 
is  considered  to  be  a  potential  cost  driver  for  two  s^sarate  reasons. 
First,  if  software  is  to  be  developed  for  reuse,  additional  cost 
iitpacts  will  be  associated  with  packaging,  increased  documentaticai  and 
stricter  reliability  requirements.  Second,  if  software  from  another 
project  is  to  be  reused  in  a  new  project,  then  the  development  cost 
should  decrecise  because  less  software  has  to  be  written.  However,  the 
costs  associated  with  managing  reusable  components  need  to  be  accounted 
for  because  reuse  cannot  be  acccnmodated  without  use  of  such  an 
infrastructure. 

Of  the  cost  models  examined,  all  of  them  account  for  the  second 
reusability  issue,  the  usage  of  reusable  software.  In  every  model, 
size  and  language  used  for  the  reusable  cojiponents  were  ascertained. 
Only  three  models  (PRICE  S,  Ada  OOCXMD,  and  SoftCOst-Ada)  however,  took 
into  account  the  first  issue,  developing  reusable  software. 

PRICE  S.  PRICE  S  offers  two  methods  for  estimating  development  of 
reusable  code.  First,  estimators  can  describe  the  increase  in 
requirements  to  produce  reusable  code  by  adjusting  the  development 
ccrplexity  by  +.1  to  +.2.  This  view  is  mainly  used  when  developers 
intend  to  use  time,  rather  than  adding  people,  to  meet  reusability 
requirements.  Second,  estimators  can  describe  reusability  as  an 
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increase  in  the  application  difficulty.  Reusable  ccnpc^ients  can  be 
grouped  separately,  with  each  grotp  assigned  its  cwn  ^jplicaticai  value. 
This  view  is  ^:prcpriate  vAien  both  resources  and  time  will  be  used  to 
achieve  reusability  [PAPK89]. 

TABLE  2-7 

ADA  CXOCMD  DEX3REE  OF  REUSE  PARAMETER 


Rating 

Rating  Scale 

Ada 

NCM 

Not  for  Reuse  Elsewhere 

1.0 

H 

Reuse  Within  Single  Mission 

1.10 

VH 

Reuse  Across  Single  Product 

1.30 

EH 

Reuse  in  Any  Application 

1.50 

! 

j 

Ada  OQGCMD.  Develcpment  of  reusable  code  is  accounted  for  in  the 
Degree  of  Reuse  (RUSE)  parameter.  With  this  input,  the  estimator 
enters  the  degree  of  reusability  for  which  the  software  is  being  built. 

The  FUSE  peurameter  Wcis  incorporated  as  an  inprovement  to  standard 
COCOMD  as  well  as  a  feature  of  Ada  CXXXMO.  Table  2-7  provides  the  RUSE 
cost  driver  ratings  and  associated  effort  multipliers. 

SoftCbst-Ada.  SoftCost-Ada  views  develcponent  of  reusable 
ccnpcaTents  in  Ada  differently  than  development  of  reusable  caipanents  ' 

in  other  languages.  Its  input  parameter,  FUSE,  is  one  of  the  four  new 
and  unique  cost  drivers  added  to  SoftCost-Ada  that  originally  was  not 
found  in  SoftC3ost-R  [RCI88].  The  main  philosc^y  behind  this  parameter 
is  that  Ada  has  specific  features,  generics,  which  have  been  included  | 

in  the  language  to  make  developing  reusable  coiponents  easier.  Further,  j 

as  explained  in  section  2.2.3,  onoe  the  developer  beccnes  more 
proficient  in  the  language,  reusable  software  will  be  even  easier  to  i 

I 

develop.  The  actual  deteminaticxi  behind  SoftCost-Ada 's  raise  factor  is  ; 


2-27 


Ada  has  several  different  features  that  could  potentially  inpact 
productivity:  packages  (collections  of  related  programs  and  data) , 
overload  cperators  (the  feature  of  giving  a  new  meaning  to  an  operator, 
useful  for  defining  arithmetic  operations  for  types  that  are  not  built 
into  Ada),  strong  typing  (the  restriction  against  mixing  data  types 
across  assignments  in  expressions),  generics  (a  method  of  overcaning 
Ada's  sometimes  overly  strong  typing),  tasks  (a  method  to  allow 
concurrent  processing) ,  and  excepticn  handlers  (a  method  to  capture 
program  errors  and  continue  processing) .  The  conplexity  of  the 
features  to  be  used  on  a  given  system  d^xends  strongly  on  the 
application  type  of  that  system  (avionics,  business,  oonmand  and 
control)  [TEC087].  For  exaitple,  a  business  system  and  a  command  and 
ccntrol  system  would  both  contain  exception  handlers  but  the  difference 
would  occur  in  the  conplexity  of  the  exception  handler.  In  a  business 
system,  the  exception  handlers  may  be  less  conplex  because  a  criticeil 
error  would  cause  only  loss  of  data.  In  a  command  and  control  system, 
the  handlers  would  be  more  cotrplex  because  a  critical  error  could  cause 
loss  of  life. 

Nbn-Ada-specific  models  do  not  differentiate  between  specific  Ada 
language  features  that  are  utilized  within  a  program.  The  coiplexity 
of  the  applicatiai  is  taken  into  account,  but  there  are  no  benefits 
vhen  using  Ada  language  features  extensively.  For  example,  packaging 
technology  may  be  utilized  to  varying  degrees  on  different  projects. 
Non-Ada-^xecific  models  do  not  provide  a  direct  means  to  evaluate  the 
Ada  packaging  technology  aspect.  With  regards  to  the  Ada-specific 
models,  Ada  features  such  as  packaging  were  accounted  for  in  the 
factors  that  reflect  the  extent  of  ccnpliance  with  the  Ada  Process 
Model  and  in  the  conplexity  parameters.  A  conparison  of  the  Software 
Product  Conplexity  factor  in  Standard  OOOCMD  versus  Ada  OOOCMO  in 
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Table  2-8  illustrates  that  Ada  features  make  many  octnplex  cx)nstructs 
more  strai^tforward  [BOEH86]. 


TABLE  2-8 

OOMEARISCW  OF  SOFIWARE  PRODUCT  OCMPIEXITY  IN  STANDARD  OOOCMD 

VERSIK  ADA  OOOCMD 


Rating 

Standard 

Ada 

VL 

0.70 

0.73 

hbhh 

0.85 

0.85 

NQM 

1.00 

0.97 

H 

1.15 

1.08 

VH 

1.30 

1.22 

EH 

1.65 

1.43 

2.2,6  Influence  of  the  Ada  Programming  Support  Environment 

The  Ada  Programming  Su^^rt  Environment  is  a  set  of  coordinated 
tools  for  the  Ada  language.  It  contains  such  software  tools  as 
[B00C87] : 


■  Text  editor 

•  P>retty  printer 

•  Oonpiler 

•  Linker 

•  Set-use  static  analyzer 

•  Control-flcw  static  analyzer 

•  Dynamic  analysis  tools 

•  Terminal  interface  routines 

■  File  administrator 

•  Coctmand  interpreter 

•  Oonfiguration  manager 

•  APSE  interface  to  underlying  machine. 
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Althou^  the  APSE  has  not  matured  to  its  fullest  potentieLL,  experts 
cited  in  the  ESD  R^xjrt  said  that  once  it  does,  the  cost  of  software 
development  will  decrease  [TBO087].  These  tools  will  enable  the 
developer  to  effectively  and  efficiently  use  the  Ada  language  resulting 
in  increased  software  productivity. 

.  Every  model  in  the  test-case  study  accounted  for  the  APSE  as  a 
cost  driver  and  instructions  are  provided  in  each  model  to  help 
determine  the  recoramended  iiput  for  the  tools  environment.  Each  model 
rewards  the  developer  for  upgrades  in  the  tool  si?:port.  The  SoftCost- 
Ada  model  was  recalibrated  to  reflect  the  inpact  of  a  workstatioiyAPSE 
strategy.  In  a  study  conducted  over  three  years,  Rd  developed  new 
calibrations  to  shew  the  effects  of  APSE  factors  on  productivity  and 
cost  [RCI89]. 

2.2.7  Lanquaae  and  Tool  Maturity 


Some  of  the  language  and  tool  maturity  issues  have  been  discussed 
in  previous  sections  (2.2.3  and  2.2.6),  but  one  topic  still  exists, 
cptimization.  The  cptimization  prc±)lem  can  be  illustrated  in  examining 
Ada  tasking.  Tasks  are  used  as  a  mechanism  for  development  of 
concurrent  software.  Recently,  this  feature  has  come  under  a  lot  of 
fire  at  Ada  Jovial  User  Groip  (AdckJUG)  meetings  because  Ada  lacks  a 
suitable  method  of  providing  regularly  scheduled  tasks  within  maximum 
time  constraints  [TECX)87].  CXirrently,  tasks  do  not  meet  embedded 
(airborne)  system  needs.  This  is  because  of  large  object  code  sizes. 
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slew  run  times,  and  prtxjessors  with  limited  memory  and  severe  time 


constraints  [TECJD87].  All  of  these  factors  are  peurt  of  the 
optimization  problem.  Eventually,  ccnpiler  writers  should  solve  the 
cptimization  problem,  but  currently  optimization  issues  have  an  inpact 
on  the  software  develcpment. 

Althou^  the  cptimization  issue  is  prominent  in  today's  software, 
only  SoftCost-Ada  handles  this  issue  through  its  degree  of  real-time 
parameter.  This  questions  essentially  asks  "How  much  Ada  tasking  is 
involved?"  Otdier  cost  models  exclude  the  issue  of  tas)dng  fron  their 
equations  with  the  assumption  that  as  technology  improves, 
cptimization,  in  terms  of  tasking,  will  no  long  be  an  issue. 


2.2.8  Hawthorne  Effect 


The  Hawthorne  Effect  encompasses  the  jhiloscphy  that  experimentaLL 
Ada  projects  may  be  unduly  influenced  by  the  "better"  personnel  vho  are 
allocated  to  the  project  and  the  fact  that  the  project  is  being  closely 
monitored  [TE0087].  Respondents  from  the  ESD  report  thought  that  this 
issue  could  be  a  potent ied  oc3st  driver. 

A  recent  article  [0PME86]  describes  eui  investigation  in  vhich 
12  sites  were  asked  hc3W  personnel  were  selected  for  Ada  projects.  More 
than  300  staff  were  allccated  in  a  ratio  of  about  60:40, 
conscript :voluntctry.  There  was  no  evidence  that  Ada  projects  were 
given  ein  undue  priority  for  resources.  Project  itanagers  always  fought 
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for  the  best  staff  available.  In  oaiclusion,  there  was  no  evidence 
that  either  supported  or  di^roved  the  idea  of  the  Hawthorne  effect. 

Ncaie  of  the  models  reviewed  consider  the  Hawthorne  effect. 
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3.0  AEPIJOffiiai  OF  SEUX3EI)  CX)CT  MODEXS  TO  ADA  HOIBCK 

3.1  BACKGROUND 

This  section  provides  an  overview  of  the  Ada  projects  targeted  in 
the  model  applications.  Ihe  methodology  used  to  collect,  validate  and 
normalize  project  data,  derive  model  inputs,  and,  sufcsequently, 
interpret  the  results  is  presented. 

3.2  OVERVIEW  OF  ADA  FROJECTS  TARGETED  IN  THE  TEST  CASE  STODY 

Data  was  aquired  for  10  Ada  projects;  however,  only  ei^t  were 
targeted  in  the  test  caise  study.  Table  3-1  provides  a  summary  of  the 
application  and  functionality  of  projects.  In  addition,  Appendix  H 
presents  an  overview  of  each  project  with  regard  to  the  following 
areas: 

•  Product  Description 

•  Software  Size 

•  Development  Process 

•  Catpxjter  System. 

These  areas  map  to  model  input  categorizations  contained  in  Appendix  B. 

3.3  MEIH0D0D3GY 

Project  data  needed  to  be  validated  and  normalized  before  it  could 
be  used  for  irput  to  the  models.  The  following  subsections  correspond 
to  activities  that  supported  the  model  afplications: 

•  Data  Collection 

•  Data  Validation 
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TABLE  3-1 


JKUECT  KJNCTKmL  DESCRIPTIONS 


PROOECT  NCMBER 


1 


.2 


3 


4 


5 


6 


7 


8 


9 


10 


CESCSOPnCN 


C(ariinerci2LL  real-time  control  center  traffic 
planning  and  monitoring  system  that  provides 
traffic  planning,  controller  interfaces,  and 
traffic  si;5)ervision. 

Tactical  conmard  and  control  system  that 
si:5:ports  planning,  directing  and  executing  of 
orders  issued  ty  the  ccmmander. 

APSE  interface  tool  that  provides  multiple 
screen  images  and  runs  a  host  operation 
system  command  interpreter  in  eaoh  window. 

Ada  interactive  environment  that  iitproves 
productivity  for  the  develcpment  of  large  Ada 
software  systems.  Provides  interactive  Ada 
conpilation  and  cross  compiler,  debugging 
tools,  and  software  management. 

Development  tool  that  integrates  design  or 
development  of  hardware  and  software  for 
embedded  systems  before  prototyping. 

Avionics  guidarnoe  and  control  system  with 
software  for  pre-programmed  flight  paths  and 
full  vp,  guided  flight. 

Command  and  control  system  that  is  designed 
to  support  testing  and  evaluation  of  taotioal 
doctrine. 

Ccmimand  and  control  system  that  performs 
target  detection,  target  recognition  and 
identification,  and  laser  target  designation 
for  a  mission  payload  subsystem. 

Component  within  an  avionics  system  that 
controls  stabilizer  position,  aileron 
Icxdcout,  and  ratio  of  rucJder  movement.  All 
firmare. 

Corraiand  and  control  system  that  will  serve  as 
both  a  fire  support  control  and  ccxDrdination 
system  and  a  field  artillery  system. 


3-2 


Normalization 


Parameter  Ratings  Selection. 


3.3.1  Data  Collection 

Projects  targeted  in  this  test  case  study  were  ciDtained  from  two 
primary  sources; 

1.  Air  Force  Electronic  Systems  Division  (ESD)  at  Hansoom  AFB. 

2.  Direct  inquiry  of  organizations  identified  in  the  AJPO  Ada 
Usage  Database. 

Four  projects  (identified  as  Projects  2,  3,  4,  and  10  in  Table  3-1) 
were  cttained  from  ESD.  One  of  these  projects  (Project  10)  was 
discarded  because  effort  expenditure  data  was  emitted  from  the  data 
collection  form. 

Inquiries  into  the  projects  listed  in  the  AJPO  Ada  Usage  Database 
resulted  in  19  respondents  vho  agreed  to  provide  data  for  this  study. 
Of  these  newly  identified  projects,  four  were  eventually  able  to 
provide  requested  data  in  a  timely  manner.  These  were  Projects  5,  6, 
7,  and  8  in  Table  3-1. 

The  two  remaining  projects  targeted  in  the  study,  Projects  1  eind 
9,  were  chtained  throuch  randan  inquiries.  One  of  these  projects 
(Project  9)  Wcis  discarded  because  of  its  size  (<  5000  SIOC) . 

Table  3-2  provides  an  overview  of  projects  that  were  targeted  in 
the  test  case  study. 
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TABLE  3-2 

PROJECT  SUMMARY  INPOPMATIOI* 


PROJECT 

NUMBER 

ACTUAL 

EFPORr 

(EM) 

TYPE  OF 
PROJECT 

TYPE  OF 
OCWTRACT 

PROJECT 

SIZE** 

(SIDC) 

Ada 

EXPERIENCE 

(YEARS) 

raiVEIDIMENT 

APPROACH 

1 

302 

Ccxnmand  & 
Control 

CcramercieLl 

50,000 

1 

(Xjject 

Oriented 

Design 

2 

684 

Conimand  & 
Control 

Government 

115,000 

1 

Ctoject 

Oriented 

Design 

3 

134 

Tool/ 

Environment 

Ccmmercieil 

18,640 

1 

CSaject 

Oriented 

Design 

4 

692 

Tool/ 

Environment 

Commercicil 

480,000 

5 

(SDject 

Oriented 

Design 

5 

144 

Tool/ 

Environment 

Commercial 

136, 000 

3 

CSaject 

Oriented 

Design 

6 

190 

Avionics 

Government 

31,800 

0 

Structured 

Design 

7 

696 

Ccmmand  & 
Control 

Government 

69,160 

0 

Structured 

Design 

8 

322 

Command  & 
Control 

Government 

18,300 

0 

Object 

Oriented 

Design 

*  Projects  9  and  10  were  discarded 

**  Size  includes  reused  and  non-Ada  code.  See  Appendix  H  for  a  software  size 
description  of  each  project. 
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3.3.2  Data  Validation 


Data  was  provided  for  each  Ma  project  in  the  form  of  ocnpleted 
project  questicxTnaires.  The  first  set  of  questionnaires  contained  data 
collected  by  Tecolote  Research  on  four  projects  using  data  collection 
forms  pr^jared  for  USAF/ESD.  The  second  set  of  questionnaires 
contained  data  collected  by  IIT  Research  Institute  (IITRI)  using  forms 
HIKE  devised  from  an  examination  of  model  input  parameters.  IITRI 's 
forms  are  an  extension  of  RCI's  Data  C3ollection  Form 
(RCI-TP-0197-FINr) . 

The  new  forms  were  developed  in  view  of  feedback  from  software 
develc^jers  providing  the  project  information  vho  pointed  out  the  ESD 
forms  were  intimidating.  Validating  the  data  contained  within  the 
forms  was  difficult  because  of  the  sheer  volume  of  information 
contained  within  them.  Further,  much  of  the  detailed  information 
requested  on  the  forms  was  not  needed  to  run  the  models.  Hence, 
significant  factors  that  inpact  a  project's  schedule  were  not  readily 
apparent. 

Inconsistencies  or  discrepancies  with  regard  to  the  contents  of 
the  project  questionnaire  were  noted  and  later  resolved  throu*^ 
subsequent  conversations  with  the  software  develcper  providing  project 
data.  S^»rate  validation  was  eilso  conducted  for  the  sensitivity 
analysis  portion  of  this  study  [REIF89].  In  several  instances  sate 
changes  to  the  ESD  data  were  reccmmended  regarding  quality  of  staffing, 
tools,  type  of  teams  used,  and  average  person-months  based  on  an 
examination  of  other  sources.  Personnel  vho  ccaipleted  the  forms  were 
contacted  to  verify  responses  on  the  project  questionnaires.  Changes 
to  the  project  data  vhich  were  reccniiEnded  by  personnel  who  validated 
the  data  were  inplemented  only  if  those  collected  the  data  agreed 
with  the  reocninaidatu.ans. 
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3.3.3  NoTTnal  i7.at--ion 


In  ortier  to  vise  the  data  for  subsequent  analysis,  project  data 
needed  to  be  in  a  ccnparable  format.  Each  software  developing 
organization  that  submitted  project  data  defined  key  productivity 
parameters  differently.  The  following  definitions  were  employed  to 
normalize  SWC,  person  months,  and  scope  of  effort  across  projects; 


An  Ada  source  line  of  code  was  defined  using  terminal 
semicolons. 

A  person  month  was  defined  to  be  160  hours  of  direct 
chargeable  labor. 

Given  the  typiccd.  development  oycle  during  full-scale 
development  shewn  in  Figure  3-1,  the  scope  of  the  effort 
provided  by  the  developers  extended  from  the  System  Design 
Review  (SCR)  to  fineil  software  aco^rtance.  Systemi 
requirements  analysis  and  system  integration  and  testing  were 
not  included  in  the  base  effort. 

Software  activities  included  in  the  effort  provided  by  each 
developer  did  not  include  Quality  Assurance  (QA) , 
Configuration  Management  (CM) ,  and  project  level  management 
functions  for  all  projects  targeted  in  the  study.  Table  3-3 
shows  the  software  cost  elements  that  eire  assumed  to  be 
covered  by  actual  effort  provided  by  each  software  developer 
providing  project  data,  in  the  terminology  of  each  model. 


Ada  Sourcp-  T.inps  of  (3ode  fASIDC) 


Five  definitions  are  currently  being  advanced  for  an  ASIDC 
[RCI88]: 


1.  Fhvsical  T.in^  -  any  carriage  return  or  line  feed  including 
comments  and  blank  lines.  Reusable  code  is  counted  the  first 
time  it  is  instantiated. 

2 .  Non-Cemment.  Non-Blank  T.ines  -  physical  lines  excluding 
comments  and  blcink  lines. 
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Software  Development  Cycle  for  Full-Scale  Development  CDoD-STD-2167A] 


TABLE  3-3 


SOFIWARE  ACTIVITIES  INCUJDED  IN  THE  ACTUAL  EFFORT  PROVIDED  BY 
EACH  SOFTWARE  ISVEIDPER  PROVIDING  PROJECT  DATA  IN  THE 
TEFMINOIOGY  OF  EACH  MODEL 


GOSDCnL 

PRICE  S 

100% 

Requirements  Analysis 

100% 

Software  Design 

100% 

Product  Design 

100% 

Programming 

100% 

Programming 

100% 

Documentation 

100% 

Test  Planning 

* 

Systems  Engineering 

100% 

Verification  and 

and  Program  Management 

Validation 

0 

QA 

0% 

Project  Office 

0 

CM 

100% 

Manuals 

SASIET 

SoftOost-Ada 

100% 

Software  Engineering 

100% 

Software  Development 

100% 

Systems  Engineering 

0 

Software  Management 

0 

QA 

0 

Softwcire  CM 

100% 

Test  Engineering 

0 

Software  Quality 

Evaluation 

SPCR/20 

SYCTEM-3 

0 

Planning 

100% 

Systems  Engineering 

100% 

Requirements 

0 

Project  Management 

100% 

Design 

100% 

Design 

100% 

Coding 

100% 

I>rogrammers 

100% 

IntegratioryTest 

0 

QA 

100% 

Documentation 

0 

CM 

0 

Management 

100% 

Test 

100% 

Data  Manipulation 

*  Note:  The  following  conversion  was  perfonned  to  discount  Project 
Management  fran  the  total  estimate  [PARK89A] : 

Systems  Engineering  (MM)  = 

((Design  cost  /  .69)  *  31%)  +  ((Coding  Cost  /  .75)  *  25%) 


I 

I 


3. 


Terminal  Semicolons  -  a  statement  terminated  by  a  semicolon, 
including  data  declarations,  code  used  to  instantiate  a 
reusable  ccnponent  and  the  reusable  ccanrponent  itself  the 
first  time  it  was  instantiated.  When  multiple  semicolons  are 
i;ised  within  a  declaration  statement,  the  terminating 
semicolon  is  used  to  define  the  termination  of  the  source 
line  of  code.  For  exaiiple,  a  package  specification  which 
included  a  statement  that  spans  ten  lines  eind  is  terminated 
by  a  single  semicolon  would  count  as  caie  ASIDC.  Comments, 
blank  lines  cind  non-deliverable  code  are  not  included  in  the 
line  count. 

4.  Essential  or  T.imitPri  Terminal  Semicolons  -  terminal 
semicolons  e»::luding  those  used  in  data  declarations  or 
formal  parameter  lists. 

5.  Body  Semicolons  -  a  statement  terminated  by  a  carriage  return 
in  the  specification  and  a  terminal  semicolon  in  the  body  of 
an  Ada  program,  inclvKJing  data  declarations  and  code  used  to 
instcintiate  a  reusable  component  itself  the  first  time  it  was 
instantiated.  Coranents,  blank  lines  and  non-deliverable  code 
are  not  included  in  the  line  count. 


An  ASLDC  was  defined  using  terminal  semicolons.  Counts  provided  by 
project  developers  using  other  definitions  were  converted  using  the 
following  conversion  factors  [RCI88]: 


Definition  Conversion  Factor 

•  Non-Comment,  Non-Blank  Lines  Reduce  count  by  20% 

■  Terminal  Semicolons  1  to  1 

•  Essenticil  Semicolons  Increase  count  by  30% 

•  Body  Semicolons  Reduce  count  by  5% 

One  of  the  projects  (Project  5)  provided  a  size  estimate  in 
fiiysiccil  lines.  After  a  discussion  with  the  developer  of  Project  5,  it 
was  decided  to  run  two  estimates:  one  assuming  that  1  of  every  5  lines 
was  commented  and  the  second  reflecting  that  1  in  10  lines  was 
ccmmented.  Line  ccunts  were  adjusted  to  account  for  ccmmented  lines. 
The  adjusted  line  count  was  reduced  by  20%  to  normalize  for  carriage 
returns. 
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3.3.4  Criteria  For  Ratims  Selection 


For  all  projects,  model  irputs  were  set  at  an  average  rating  if 
there  was  no  data  to  support  deviating  from  the  nominal  value.  There 
were  instances,  however,  vhen  a  model  input  parameter  had  no  associated 
nominal  rating  or  information  provided  on  the  forms  was  disregarded. 
One  instance  was  with  regard  to  the  project's  required  development 
schedule.  For  consistency,  this  factor  was  assumed  to  be  nominal 
regardless  of  the  developer's  assessment.  Other  criteria  pertain  to 
specific  models. 

PRICE  S 


Schedule.  The  schedule  iqput  for  PRICE  S  requires  that  the  System 
Design  Review  (SER)  or  Software  Requirements  Review  (SRR)  be  specified. 
These  review  dates  were  not  provided  for  any  of  the  projects  targeted 
in  the  study.  Therefore,  it  was  assumed  that  the  date  of  SRR  would 
feill  on  the  date  vhich  corresponded  to  one  fourth  of  the  entire 
development  schedule. 

Productivity  Ehctor.  The  productivity  factor  is  an  eitpirically  derived 
value  that  describes  the  demonstrated  performance  of  each  develc^ing 
organization.  This  value  was  assumed  to  be  nominal. 

COnplexity.  The  complexity  adjustment  for  a  new  language  was  assumed 
+.1  since  the  this  factor  could  range  from  0.0  to  0.2. 

Source  Code  Mix.  Non-ESD  project  data  did  not  contain  a  breakdown  of 
the  software  profile.  The  mix  element  application  value  for  these 
projects  was  assumed  to  be  the  value  associated  with  the  application 
type  of  the  software. 


SoftCost-Ada 

Ada  Projects  Cbnpleted.  The  response  to  the  number  of  Ada  projects 
completed  by  the  develcpment  team  was  incremented  by  one  if  an  Ada  PDL 
was  used  during  the  develcpinent. 

Software  Organizations  Involved.  Projects  2,  3,  and  4  did  not  sipply 
the  number  of  organizations  providing  level  of  effort  support  (i.e.  CM 
and  QA) .  Development  contractors  were  assumed  to  have  three  software 
organizations . 
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3.4  RESUnrS 


Section  3.4.1  presents  the  results  of  the  model  applications  that 
required  some  normalization.  Tables  are  provided  to  illustrate  the 
relative  difference  between  estimated  eind  actual  effort  for  each 
project.  Sections  3.4.2  and  3.4.3  conpare  the  performances  of  models 
in  the  areas  of  accuracy,  consistency,  contract  type,  application  type, 
and  Ada  issues. 

3.4.1  Normal i 7ati nn 


Model  outputs  were  subjected  to  a  nomalization  process  so  that 
results  would  reflect  the  same  scope  of  coverage.  This  process 
consisted  of  subtracting  model  efforts  that  were  associated  with  phases 
not  included  in  the  actual  effort  (effort  that  was  prior  to  SCR  and 
subsequent  to  the  software  acc^jtance  review) .  In  addition,  Quality 
Assurance,  Configuration  Management,  and  Program  Management  efforts 
were  also  subtracted  because  these  organizations  were  not  part  of  the 
effort  provided  (Refer  to  Section  3.3.3,  Normalization).  Table  3-4  is 
provided  to  illustrate  the  normalization  process  that  was  applied  to 
each  model. 


TABIE  3-4 

NOPMALEZATiai  OF  PRCOECT  1  EFFORT 


MODEL:  OOSIMODL 

MM 

Normalization 

Normalized  MM 

Requirements  Analysis 

17.8 

100% 

17.8 

Product  Design 

33.1 

100% 

33.1 

Programming 

93.4 

100% 

93.4 

Test  Planning 

13.2 

100% 

13.2 

Verification  &  Valid 

31.6 

100% 

31.6 

Project  Office 

19.7 

0% 

0 

CH/QA 

15.3 

0% 

0 

Manucils 

16.1 

100% 

16.1 

TOTAL  EFFORT: 

240.2 

NORMALIZED  EFFORT:  205.2 

TABLE  3-4  (Continued) 
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TABIE  3-4  (Continued) 


MODEL:  SYSTEM-3 

MM 

Normalization 

Normcilized  MM 

System  Engineering 

48 

100% 

48 

Project  Management 

35.7 

0% 

0 

Design 

141.8 

100% 

141.8 

Programmers 

148.8 

100% 

148.8 

Quality  Assurance 

25.1 

0% 

0 

Configuration  Mgt 

23.3 

0% 

0 

Test 

110.3 

100% 

110.3 

Data  Manipulation 

11.5 

100% 

11.5 

TOTAL  EFFORT: 

544.5 

NORMALIZED  EFFORT:  460.4 

MODEL:  SASET 

MM 

Normalization 

Normalized  MM 

Software  Engineering 

307.59 

100% 

307.59 

Systems  Engineering 

53.76 

100% 

53.76 

Q.A. 

23.31 

0% 

0.00 

Test  Engineering 

62.18 

100% 

62.18 

TOTAL  EFFORT: 

446.84 

NORMALIZED  EFFORT:  423.53 

After  normalization,  model  estimates  were  crmpared  to  ac±ual 
effort  data  to  determine  the  relative  error. 

In  three  instances,  models  were  not  used  to  derive  project 
estimates.  The  C30SIM0DL  inplementation  of  IOC  Ada  OOOCM)  provides 
equations  that  af^ly  to  the  OOCCMD  embedded  mode  of  software 
development.  Ihe  model  cx>-develqper.  Dr.  Barry  Boehm,  confirmed  that 
equations  currently  do  not  exist  for  the  organic  and  semidetached 
modes.  Two  of  the  projects  targeted  in  the  test  Ccise  stucfy.  Projects  3 
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and  4,  fall  into  the  semidetached  category  of  software  develcpnnent. 
Thus,  CXJSTMDDL  w^ls  not  applied  to  these  projects. 

Project  4  was  used  by  Rd  for  the  develcpnent  of  the  SoftCost-Ada 
model.  Therefore,  so  as  not  to  ccnpronise  credibility  of  the  test  case 
study  results.  Project  4  Wcis  not  targeted  for  application  to  SoftCost- 
Ada. 


3.4.2  CCmparison  of  Model  Results 

As  stated  in  the  c±»jective,  model  performances  were  analyzed  with 
regard  to  five  areas:  overall  accuracy,  overall  consistency,  contract 
type,  application  type,  and  Ada  issues.  Model  performances  vctried  with 
regard  to  each  assessment  criteria.  The  following  paragraphs  specify 
the  findings  for  each  of  these  areas. 

3.4.2. 1  Overall  Accuracy 


Model  effort  and  schedule  projections  were  ccsrpared  to  act\3l 
effort  and  schedule  expended  by  the  project,  develc^jer  to  ascertain 
vtiich  models  provided  the  most  accurate  results.  Results  in  view  of 
projected  effort  versus  actual  effort  are  provided  in  Tables  3-5  and 
3-6.  A  cortparison  of  projected  schedules  versus  actual  schedules  are 
illustrated  in  Tables  3-7  cind  3-8.  'Ihe  number  of  effort  estimates  that 
were  accurate  within  +30%  were  cis  follows: 


MODEL 

PERFORMANCE 

RANGE 

1. 

SoftCost-Ada 

4  out  of  7 

0%  to  13% 

2. 

SASET 

4  out  of  8 

-29%  to  29% 

3. 

SPQIV20 

3  out  of  8 

-22%  to  19% 

4. 

COSTMODL 

2  out  of  6 

-25%  to  -1% 

5. 

PRICE  S 

0  out  of  8 

Not  Afplicable 

6. 

SySTEM-3 

0  out  of  8 

Not  T^licable 

3-14 


I 

I 

I 


TABIZ  3-5 

MODEL  EFFORT  FROJECITONS 


Number 

Actual 

Effort 

SoftCost- 

Ada 

OOSIMDDL 

PRICE  S 

SASET 

SYSTEM-3 

SPQR/20 

1 

302 

319 

205 

788 

424 

460 

237 

2 

684 

1446 

516 

1487 

635 

2258 

811 

3 

134 

137 

Not  Run 

42 

95 

90 

296 

4 

692 

Not  Run 

Not  Run 

2223 

1931 

2323 

2876 

5 

144 

144 

356 

985 

725 

1366 

260 

6 

190 

360 

93 

879 

335 

641 

109 

7 

696 

1621 

692 

1827 

822 

2303 

560 

8 

322 

364 

108 

623 

416 

728 

86 

TABLE  3-6 

RELATIVE  ERRORS  FDR  MODEL  EFFORTS  WHEN  COMPARED  TO  ACTUAL  EFFORTS 


Number 

Actual 

Effort 

SoftCost- 

Ada 

OOSIMODL 

PRICE  S 

SASET 

SYSTEM-3 

SPQIV20 

a 

302 

6% 

-32% 

161% 

40% 

52% 

-22% 

2 

684 

111% 

-25% 

117% 

-  7% 

230% 

19% 

3 

134 

2% 

Not  Run 

-69% 

-29% 

-33% 

121% 

4 

692 

Not  Run 

Not  Run 

221% 

179% 

236% 

316% 

5 

144 

0% 

147% 

584% 

403% 

849% 

81% 

6 

190 

89% 

-51% 

363% 

76% 

237% 

-43% 

7 

696 

133% 

-  1% 

163% 

18% 

231% 

-20% 

8 

322 

13% 

-66% 

93% 

29% 

126% 

-73% 
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TABIZ  3-7 


Actual 

Schedule 

SoftCost- 

Ada 

OOSIMDDL 

ERICE  S 

SASET 

SYSTEM-3 

SPQR/20 

27 

26.04 

20.3 

50.0 

28.0 

25.12 

33.9 

38 

61.48 

27.4 

45.0 

28.7 

31.45 

57.6 

35 

17.39 

Not  Run 

20.0 

15.1 

13.34 

57.3 

49 

Not  Run 

Not  Run 

29.0 

51.9 

43.11 

65.6 

12 

16.54 

23.5 

46.0 

38.2 

36.34 

38.7 

40 

29.52 

14.3 

47.0 

18.0 

29.09 

33.6 

62 

85.16 

28.5 

90.0 

30.4 

41.90 

56.4 

56 

25.66 

16.0 

58.0 

22.4 

28.19 

34.6 

TABLE  3-8 

RELATIVE  ERRORS  FOR  MODEL  SCHEDULES  WHEN  CCKPARED  TO  ACTUAL  SCHEDULES 


Proj . 
Number 

Actual 

Schedule 

SoftCost- 

Ada 

OOSTMODL 

PRICE  S 

SASET 

SYSTEM-3 

SPQIV20 

■ 

27 

-  4% 

-25% 

85% 

4% 

-7% 

26% 

2 

38 

62% 

-28% 

18% 

-24% 

-17% 

52% 

3 

35 

-50% 

NR 

-43% 

-57% 

-62% 

64% 

4 

49 

NR 

NR 

-41% 

6% 

-12% 

34% 

5 

12 

38% 

95% 

283% 

218% 

203% 

223% 

6 

40 

-26% 

-64% 

18% 

-55% 

-27% 

-16% 

7 

62 

37% 

-54% 

45% 

-51% 

-32% 

-  9% 

8 

56 

-54% 

-71% 

3% 

-60% 

-50% 

-38% 
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The  nvnnber  of  schedule  estimates  that  were  accurate  within  +30%  were  as 
follows ; 


MODEL 

PERPOFMANCE 

RANGE 

1. 

SYSTEM-3 

4  out  of  8 

-27%  to  -7% 

2. 

FKECE  S 

3  out  of  8 

3%  to  18% 

3. 

SASET 

3  out  of  8 

-24%  to  6% 

4. 

SPQEV'20 

3  out  of  8 

-16%  to  26% 

5. 

OQSTMODL 

2  out  of  6 

-28%  to  -25% 

6. 

SoftOost-Ada 

2  out  of  7 

-26%  to  -4% 

3 . 4 . 2 . 2  Overall  Consistency 

Since  the  objective  of  the  study  was  to  ccsipeure  model  performanoe 
vhen  applied  to  the  same  project  data,  special  enphasis  was  placed  on 
obtaining  consistent  interpretations  of  project  data  across  all  models. 
To  aocomplish  this  task,  one  person  was  ohosen  to  derive  all  inputs  for 
all  projects.  This  insured  that  the  same  knowledge  base  was  used  to 
interpret  projecrt  information  arx3  model  parameters  when  inputs  were 
derived.  After  results  were  obtained,  an  analysis  of  model  efforts  was 
performed  to  establish  if  results  were  consistently  hi^  or 
consistently  low,  eliminating  differences  between  the  perspectives  of 
the  person  deriving  the  inputs  and  the  model  developer.  This  process 
involved  the  following  steps: 

1.  A  percentage  of  actual  effort  to  model  effort  was  calculated. 

2.  The  two  extremes  were  discarded  to  achieve  a  truer  saitpling 
of  percentages. 

3.  A  mean  value  of  the  remaining  percentages  was  computed  and 
applied  to  the  given  model's  estimates. 

4.  The  relative  error  for  each  project  was  recalculated  using 
the  adjusted  efforts. 

The  results  of  this  process  vhen  applied  to  each  model  are  illustrated 
in  Tables  3-9  and  3-10.  After  application  of  the  means,  the  number  of 
adjusted  effort  estimates  within  +-30%  were  as  follows; 
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TABIE  3-9 


RELATIVE  ERRORS  FOR  EFFORT  AFTER  APPLYING  MEANS 


SoftCost-Ada 

Mean=0.87 

OOSTMODL 

Mean=1.46 

EKLCE  S 
Mean=0.38 

SASET 

MeanF=0.72 

SYSTEM-3 

Mean=0.38 

SPQJV20 

MBan=1.02 

-  8% 

-  1% 

-  1% 

1% 

-  42% 

-  20% 

84% 

10% 

-  17% 

-  33% 

25% 

21% 

-  11% 

Not  Run 

-  88% 

-  49% 

-  74% 

125% 

Nc3(t  Run 

Not  Run 

22% 

101% 

28% 

324% 

-  13% 

261% 

160% 

263% 

260% 

84% 

65% 

-  29% 

76% 

27% 

28% 

-  41% 

103% 

45% 

0% 

-  15% 

26% 

-  18% 

-  2% 

-  51% 

-  26% 

-  7% 

-  14% 

-  73% 

TABLE  3-10 

RELATIVE  ERRORS  FOR  SCHEDULE  AFTER  APPLYING  MEANS 


SoftCi3st-Ada 

Mean=0.90 

(OOSTMODL 

Mean=1.93 

PRICE  S 
Mean=0.69 

SASET 

Mean=1.63 

SYSTEM-3 

MeaiT=1.38 

SPQIV'20 

Mean=0.85 

-13% 

45% 

28% 

69% 

28% 

7% 

46% 

39% 

-  18% 

23% 

14% 

29% 

-  55% 

Not  Run 

-  61% 

-  30% 

-  47% 

39% 

Not  Run 

Not  Run 

-  59% 

73% 

21% 

14% 

24% 

278% 

165% 

419% 

318% 

174% 

-  34% 

-  31% 

-  19% 

-  27% 

0% 

-  29% 

24% 

-  11% 

0% 

-  20% 

7% 

-  23% 

-  59% 

-  49% 

-  29% 

-  35% 

-  31% 

-  47% 
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MODE?. 

PERFORMANCE 

RANGE 

1.  SySTEM-3 

5  cut  Of  8 

-14%  to  28% 

2.  FREGE  S 

5  out  of  8 

-26%  to  22% 

3 .  SoftCost-Ada 

4  out  of  7 

-13%  to  -2% 

4.  OOSTMODL 

3  out  of  6 

-29%  to  10% 

5.  SASET 

4  out  of  8 

-15%  to  27% 

6.  SPQEVSO 

3  out  of  8 

-20%  to  21% 

After  application  of  the 

estimates  within  +30%  were 

means,  the  nuittoer  of 

eis  follows: 

adjusted  schedule 

MODEL 

PERPOFMANCE 

RANGE 

1.  SYSTEM-3 

5  out  of  8 

0%  to  28% 

2.  PRICE  S 

5  out  of  8 

-29%  to  28% 

3.  SPQIV20 

5  out  of  8 

-29%  to  29% 

4.  SASET 

4  out  of  8 

-30%  to  23% 

5 .  SoftCost-Ada 

3  out  of  7 

-13%  to  24% 

6.  OOSTMODL 

1  out  of  6 

-11% 

3. 4. 2. 3  Analysis  By  Afplication  Type,  Contract  Type,  and  Ada  Issues 


The  last  three  areas  in  Vihich  model  performances  were  evaluated 
consisted  of  the  follcwing:  contract  type,  application  type,  and  Ada 
issues.  Table  3-2  provides  an  overview  of  projects  with  regard  to 
these  evaluation  criteria.  Performance  in  view  of  remaining  criteria 
was  evaluated  by  ccroparing  accuracy  and  consistency  of  effort 
estimates.  Findings  are  noted  belcw. 


Govemnent  Versus  Cbmnercial  Oontracts.  As  illustrated  in  Table  3-2, 
four  government  contracts  and  four  ccmmercial  contracts  were  targeted 
in  the  test  case  study.  Examination  of  model  performances  identified 
the  following  trends: 


GOVERNMENT  OOTTRACIS 

Fbdel  Accuracy.  SASET  provided  more  accurate  results  for 
government  contracts  where  three  of  four  estimates  were  within  29% 
of  the  actucLl  effort.  The  number  of  effort  estimates  accurate 
with  +30%  were  cis  follows: 
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>»ai 


RANGE 


PERFORMANCE 


1.  SASET 

2.  CXDSOMDDL 

3.  SPQIV20 

4 .  SoftCost-Ada 

5.  PRICE  S 

6.  SYSTEM-3 


3  out  of  4 
2  out  of  4 
2  out  of  4 
1  out  of  4 
0  out  of  4 
0  out  of  4 


-  7%  to  29% 
-25%  to  -1% 
-20%  to  19% 

13% 

Not  Applicable 
Not  i^licable 


Model  C3Gnsistency’.  SYSTEM-3  provided  the  most  accurate  results 
for  government  contracts  with  four  of  four  estimates  accurate 
within  28%  after  a  mean  was  applied.  The  number  of  effort 
estimates  accurate  within  ±30%  were  as  follows: 


MODEL 

PERFORMANCE 

RANGE 

1. 

SYSTEM-3 

4  out  of  4 

-14%  to 

28% 

2. 

PRICE  S 

3  out  of  4 

-26%  to 

0% 

3. 

SASET 

3  out  of  4 

-15%  to 

27% 

4. 

SPQR/20 

2  out  of  4 

-18%  to 

21% 

5. 

COSTMODL 

2  out  of  4 

-29%  to 

10% 

6. 

SoftCost-Ada 

1  out  of  4 

-2% 

OO^MERCaAL  OONTRACTS 

Model  Acx:uracy.  SoftCost-Ada  was  extremely  accurate  for 
estimating  ccinmercial  contracts.  Three  of  three  estimates  were 
within  6%  of  actual  effort.  The  number  of  effort  estimates 
accurate  with  ±30%  were  as  follows : 


MODEL 


PERFORMANCE 


RANGE 


1.  SoftCost-Ada 

2.  SPQEV20 

3.  SASET 

4.  COSTMODL 

5.  PRICE  S 

6.  SYSTEM-3 


3  out  of  3 
1  cut  of  4 
1  out  of  4 
0  out  of  2 
0  out  of  4 
0  out  of  4 


0%  to  6% 

-22 

-29 

Not  Applicable 
Not  Applicable 
Not  Applicable 


Mcxiel  consistency.  Softcost-Ada  provided  the  most  accurate 
results  for  conmercial  contracts  with  three  of  three  estimates 
within  13%  of  actual  effort  after  a  meein  was  applied.  The  number 
of  effort  estimates  accurate  within  ±30%  were  as  follows: 
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MCa®L 

PERPOFMANCE 

RANGE 

1. 

SoftCost-Ada 

3  out  of  3 

-13%  to  -8% 

2. 

FRIGE  S 

2  out  of  4 

-  1%  to  22% 

3. 

OOSTMDDL 

1  out  of  2 

-  1% 

4. 

SASET 

1  out  of  4 

1% 

5. 

SPQIV20 

1  out  of  4 

-20% 

6. 

SYSTEM-3 

1  out  of  4 

28% 

^plication  'IVpe.  Project  data  provided  by  software  developers 
consisted  of  three  different  types  of  applications:  oanmand  and 
control  (four  projects) ,  tools/environment  (three  projects) ,  and 
avionics  (one  project) .  Performances  of  models  with  regard  to  these 
application  areas  were  analyzed.  Results  were  inconclusive  with  regeud 
to  the  avionics  application  type  because  there  was  only  one  project  of 
this  type  targeted  in  the  study.  Results  are  as  follows: 


OCMWO  AND  CPNTROL 

Itodel  AacurcK^.  Estimates  generated  by  SASET  and  SPQR/20 
were  most  cottparable  to  the  actual  effort  with  three  of  four 
estimates  within  30%  of  actual  effort  for  command  eind  control 
type  projects.  Ihe  number  of  effort  estimates  accurate 
within  +30%  were  as  follows: 


MODEL 

PERPOFMANCE 

RANGE 

1. 

SASET 

3  out  of  4 

-  7%  to  29% 

2. 

SPQR/20 

3  out  of  4 

-22%  to  19% 

3. 

SoftCost-Ada 

2  out  of  4 

6%  to  13% 

4. 

C30STMDDL 

2  out  of  4 

-25%  to  -1% 

5. 

PRICE  S 

0  out  of  4 

Not  Applicable 

6. 

SYSTEM-3 

0  out  of  4 

Not  ^plicable 

Model  Consistency.  Although  all  of  the  models  did  very  well, 
FKECE  S  provided  more  accurate  results  with  four  out  of  four 
estimates  within  26%  of  the  actueil  effort  after  a  mean  was 
applied.  The  nurttoer  of  effort  estimates  accurate  within  +30%  were 
as  follows: 
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PERFORMANCE 

RANGE 

1. 

ERICE  S 

4  out  of  4 

-26%  to 

0% 

2. 

SASEI 

3  out  of  4 

-15%  to 

1% 

3. 

SYSTEM-3 

3  out  of  4 

-14%  to 

26% 

4. 

SPQIV20 

3  out  of  4 

-20%  to 

21% 

5. 

SoftCost-Ada 

2  out  of  4 

-  8%  to 

-2% 

6. 

C30STM0DL 

2  out  of  4 

-  1%  to 

10% 

TOOlS/ENVTROlMENr 

Model  Accuracy.  SoftCost-Ada  was  accurate  within  2%  on  two  exit  of 
the  two  tools/environment  projects  to  which  it  was  e^lied.  Ihe 
number  of  effort  estimates  accurate  within  +30%  were  as  follows: 

MODEL 

PERFORMANCE 

RANGE 

1.  SoftCost-Ada 

2.  SASET 

3.  OOSIMODL 

4.  PRICE  S 

5.  SYSTEM-3 

6.  SPQIV20 

2  out  of  2 

1  out  of  3 

0  out  of/  1 

0  out  of  3 

0  out  of  3 

0  out  of  3 

0%  to  2% 

-29% 

Not  ^jplicable 

Not  Applicable 

Not  Applicable 

Not  Applicable 

Model  Consistency.  SoftCost-Ada  also  provided  the  roost  accurate 
results  in  this  category  with  two  out  of  two  estimates  within  13% 
of  the  actucil  effort  after  a  mecin  was  applied.  The  number  of 
effort  estimates  accurate  within  +30%  were  as  follcws: 

MODEL 

PERFORMANCE 

RANGE 

1.  SoftCost-Ada 

2.  SYSTEM-3 

3.  PRICE  S 

4.  COSIMDDL 

5.  SASET 

6.  SPQIV'20 

2  out  of  2 

1  out  of  3 

1  out  of  3 

0  out  of  1 

0  cut  of  3 

0  out  of  3 

-13%  to  -11% 

28% 

22% 

Not  ^plicable 

Not  Applicable 

Not  ^plicable 

Ccofiarisan  of  Aia  Issues.  Model  philosophies  regarding  Ada  issues  were 
examined  to  identify  any  pxDssible  reasons  for  differences  between  model 
estimates.  An  evaluation  of  project  data  in  light  of  Ada  issues 
described  in  Section  2.2  shewed  that  six  of  the  eii^t  issues  could  ixst 
be  evaluated  for  either  of  two  reasons; 
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1.  Project  data  was  not  provided  that  could  be  used  to  evaluate 
the  issue. 

2.  When  information  was  provided,  differences  in  project  data 
with  regard  to  the  issue  were  not  apparent. 

Ihree  issues  were  ejoiluded  from  evaluation  because  of  a  lack  of  data: 
phase  distribution  of  effort,  Icinguage  and  tool  maturity,  and  the 
Hawthorne  effect.  Issues  excluded  because  of  a  lack  of  definitive  data 
were  the  reusability  iirpact,  influence  of  Ada  language  features,  and 
influence  of  the  APSE.  The  analysis  of  the  remaining  issues,  modern 
software  development  practices  and  the  learning  curve,  was  performed  by 
evaluating  model  performances  based  on  the  design  approach  and  Ada 
experience  of  the  project  developers. 

Modern  Software  Developmait  Practices:  Design  Appiroach.  Of  the  ei^t 
projects  targeted  in  the  test  case  study,  two  (Projects  6  and  7) 
utilized  a  structured  design  afproach,  and  six  used  an  OOD  methodology. 
Results  were  largely  inclusive  when  model  performances  were  evaluated 
for  design  methodology.  It  was  interesting  to  note  however  that  the 
Ada-specific  model,  Softcost-Ada,  which  was  most  accurate  on  projects 
in  vhich  OOD  was  the  specified  design  methodology  (four  out  of  five 
results  were  within  13%) ,  had  no  effort  estimates  within  30%  of  project 
actuals  on  the  two  projects  which  utilized  a  structured  design 
approach.  Given  the  small  sanpling,  it  is  difficult  to  attribute  these 
results  to  the  specified  design  methodology. 

STRDCnJRED  DESIQI 

Model  Accuracy.  Results  were  inconclusive  with  regard  to  design 
approach.  The  number  of  effort  estimates  accurate  within  +30% 
were  as  follows: 
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MODEL 

PERTORMANCE 

RANGE 

1. 

OOSTMODL 

1  out  of  2 

-  1% 

2. 

SASET 

1  out  of  2 

18% 

3. 

SPC3V20 

1  out  of  2 

-20% 

4. 

SoftCost-Ada 

0  out  of  2 

Not  Applicable 

5. 

FRIGE  S 

0  out  of  2 

Not  Applicable 

6. 

SYSTEM-3 

0  out  of  2 

Not  Applicable 

Model  Gtxisistency.  No  trends  cxMld  be  identified  given  the  small 
sanpling  of  data.  The  number  of  effort  estimates  acaourate  within 
±30%  after  means  were  applied  were  as  follows: 


MODEL 

FERFORMANGE 

RANGE 

1.  SYSTEM-3 

2  out  of  2 

26%  to  28% 

2.  SASET 

2  out  of  2 

-15%  to  27% 

3.  FRIGE  S 

1  out  of  2 

0% 

4.  SPQFV20 

1  out  of  2 

-18% 

5.  OOSTMODL 

1  out  of  2 

-29% 

6.  SoftCost-Ada 

0  out  of  2 

Not  Applicable 

OBJEGT  ORIENTED  DESIGN 

Model  Aocxiracy. 

SoftCost-Ada  was  most 

accurate  on  projects  in 

which  OOD  was  the 

specified  design  methodology.  The  number  of 

effort  estimates  accurate  within  ±30%  were  as  follows: 

MODEL 

FERFORMANGE 

RANGE 

1.  SoftCost-Ada 

4  out  of  5 

0%  to  13% 

2.  SASET 

3  out  of  6 

-29%  to  29% 

3.  SPQIV20 

2  out  of  6 

-22%  to  19% 

4.  OOSTMODL 

1  out  of  4 

-25% 

5.  FRIGE  S 

0  out  of  6 

Not  Applicable 

6.  SYSTEM-3 

0  out  of  6 

Not  Applicable 

Model  OcnsistencY. 

Results  could  not  be 

correlated  to  the  modem 

software  develc^ment  practices  language  considerations  of  the 
models  (discussed  in  section  2.2.2).  The  number  of  effort 
estimates  accurate  within  ±30%  after  means  were  applied  were  as 
follows: 
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lyODEL 


PERTORMftNCE 


RANGE 


1.  SoftC3ost-Ada 

2.  FRIGE  S 

3.  OOSIMODL 

4.  SYSTEM-3 

5.  SASET 

6.  SPQFV20 


4  out  of  5 
4  out  of  6 

2  out  of  4 

3  out  of  6 
2  out  of  6 
2  out  of  6 


-13%  to  -  2% 
-26%  to  22% 

-  1%  to  -10% 
-14%  to  28% 

-  7%  to  -  1% 
-20%  to  21% 


Teaming  curve;  Ada  E;^)eriencs.  Of  the  ei(^t  projec±s  targeted  in  the 
test  case  study,  two  (Projects  4  and  5)  professed  an  average  Ada 
experience  of  greater  than  three  years.  An  evzduation  of  model 
performances  based  on  Ada  experience  yielded  results  that  could  not  be 
correlated  to  the  learning  curve  language  considerations  of  the  models 
applied  in  the  case  study.  Results  are  provided  for  Ada  experience 
less  than  three  years  and  greater  tt?  ‘•hree  years,  respectively. 


TRqfi  TOAN  3  YEARS  OF  ADA  EXPERIRNCR 


Model  Accuracy.  No  trends  could  be  identified  in  light  of  the 
language  considerations  of  the  models.  Ihe  number  of  effort 
estimates  accurate  within  ±30%  were  as  follows; 


MODEL 


PERFORMANCE  RANGE 


1.  SASET 

2 .  SoftCost-Ada 

3 .  SPQR/20 

4.  OOSIMODL 

5.  FRIGE  S 

6.  SYSTEM-3 


4  out  of  6 
3  out  of  6 
3  out  of  6 
2  out  of  5 
0  out  of  6 
0  out  of  6 


-29%  to  29% 

2%  to  13% 
-22%  to  19% 
-25%  to  2% 
Not  Applicable 
Not  Applicable 


Model  Obnsistenciy.  Results  were  consistent  and  could  not  be 
correlated  to  the  language  considerations  of  the  models.  Ihe 
number  of  effort  estimates  accurate  within  ±30%  after  means  were 
applied  were  as  follows; 


MODEL 

PERFORMANCE 

RANGE 

1. 

FRIGE  S 

4  out  of  6 

-26%  to 

0% 

2. 

SASET 

4  out  of  6 

-15%  to 

27% 

3. 

SYSTEM-3 

4  out  of  6 

-14%  to 

28% 

4. 

OOSIMODL 

3  out  of  5 

-29%  to 

10% 

5. 

SoftCost-Ada 

3  out  of  6 

-  2%  to 

-11% 

6. 

SPQfV^20 

3  out  of  6 

-20%  to 

21% 
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GraaTER  THAN  3  YEARS  OF  ADA  EXPERIENCE 


Model  Aocuracy. 

SoftCost-Ada  was  the  only  model  acxaarate 

within 

the  30%  range  on 

one  of  the  two  project 

with  personnel 

having 

greater  than  three  years  Ada  experience, 
estimates  within  +30%  were  as  follows: 

The  numbers  of 

effort 

MODEL 

PERFORMANCE 

RANGE 

1 .  SoftCost-Ada 

1  out  of  1 

0% 

2.  OOSTMDDL 

0  out  of  1 

Not  Applicable 

3.  HRICE  S 

0  out  of  2 

Not  Applicable 

4.  SYSTEM-3 

0  exit  of  2 

Not  Applic:able 

5.  SASET 

0  out  of  2 

Not  Applicable 

6.  SK2V'20 

0  out  of  2 

Not  Applicable 

Model  Oonsistency.  After  iteans  were  applied  to  effort  results, 
three  models  demonstrated  performance  within  the  +30%  range: 


MODEL 

PERFORMANCE 

RANCSE 

1. 

SoftCost-Ada 

1 

exit  of 

1 

-13% 

2. 

PRICE  S 

1 

out  of 

2 

22% 

3. 

SYSTEM-3 

1 

out  of 

2 

28% 

4. 

CJOSTMODL 

0 

out  of 

1 

Not  Afiplicable 

5. 

SASET 

0 

out  of 

2 

Not  Applicable 

6. 

SPQR/20 

0 

out  of 

2 

Not  Applicable 

3.4.3  Ccancarison  of  Nominal  Results 


Models  were  applied  using  nominal  (average)  values  for  input 
ratings  while  providing  actual  project  values  for  model  iiput 
parameters  that  must  be  estimated  early  in  the  life  cycle  and  for  which 
there  is  no  associated  average  value.  The  ncminal  iiputs  reflect  the 
level  of  knowledge  about  a  new  develcpuent  prior  to  contract  award. 
Parameters  eissumed  to  be  available  at  contract  award  included  the 
following: 
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1.  Type  of  AfplicatioiVPn3ject  Type 

2.  Estimate  Scc^  (i.e.,  ccnponent  within  a  system,  stand-alone 
program,  release,  etc.) 

3.  Project  Class 

4 .  Programming  language 

5.  System  Architecture  * 

6 .  Schedule  * 

7 .  Size  * 


(*  Although  size,  schedule,  and  system  architecture  would  not  be  kncwn 
at  contract  award,  an  estimate  would  still  be  needed.  Iherefore  the 
size,  schedule,  and  system  curchitecture  were  estimated  as  the  actual 
values  used  in  the  system.)  For  instances  v^en  ncminal  ratings  were 
not  provided,  parameters  were  examined  and  a  ncmincil  rating  was  chosen. 
The  following  assumed  values  pertain  to  specific  models  for  Vstiich  an 
average  Vcilue  was  not  clearly  identifiable: 


Ada  OOOOMO 

Experi^ioe  with  Ada  Process  Model.  The  nominal  response  to  this 
parameter  was  assumed  to  be  some  familiarity  with  practices. 

Design  Thoroughness  at  PDR.  The  nominal  response  to  design 
thoroughness  at  PDR  was  assumed  to  be  Often  (60%) . 

Risks  Eliminated  at  PER.  The  nominal  response  was  assumed  to  be 
Often  (60%) . 

Requirements  Volatility  during  Develcpnait.  The  ncminal  response 
to  requirements  volatility  during  development  was  eissumed  to  be 
occasioral,  moderate  changes. 


So ftCost-Ada 

NuEDber  of  ^3a  Projects  Ccopleted  by  Team.  The  nominal  rating 
associated  with  the  number  of  Ada  projects  coipleted  by  the 
develc^ment  team  Wcis  aissumed  to  be  1. 

SASET 


Percent  of  Memory  Utilized.  The  ncminal  value  for  percent  of 
memory  utilized  was  assumed  to  be  50%. 

Development  locations.  The  number  of  develcpment  locations  was 
assumed  to  be  1  location. 
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As  in  the  previcxis  section,  naninal  results  were  examined  with  regard 
to  accuracy,  consistency,  contract  type,  and  application  type.  An 
aneilysis  of  res>i3.ts  in  light  of  Ada  issues  was  not  performed  because 
model  iiputs  that  reflected  the  leeuning  curve  and  modem  develcpment 
practices  were  set  to  their  nomineLl  defaiHt  vadues.  The  results  cure 
presented  in  the  following  subsections. 


3 . 4 . 3 . 1  Overall  Accuracy 

Naninal  efforts  and  schedules  were  coipared  to  actual  efforts  and 
schedules  to  ascertain  vdiich  models  provided  the  most  accurate  results 
on  naninal  inputs.  Tables  3-11  and  3-12  shew  the  ncminal  efforts  and 
relative  errors  associated  with  the  six  models.  Ihe  number  of  naninal 
effort  estimates  accurate  within  +30%  were  as  follows: 


MODEL 

PERFORMANCE 

RANGE 

1. 

SASET 

4  out  of  8 

-24%  to 

29% 

2. 

SYSTEM-3 

3  out  of  8 

-17%  to 

28% 

3. 

OOSTMODL 

2  out  of  6 

-25%  to  - 

-24% 

4. 

SoftCost-Ada 

2  out  of  7 

-27%  to 

14% 

5. 

PRICE  S 

2  out  of  8 

-14%  to 

-8% 

6. 

SPQIV20 

1  out  of  8 

-27% 

Noniral  schedule  estimates  and  associated  relative  errors  are  shown  in 
Tables  3-13  and  3-14.  The  number  of  schedule  estimates  accurate  within 
+30%  were  as  follcws: 


MODEL* 

PERFORMANCE  RANGE 

1. 

SPC2iV20 

6  out  of  8 

-23% 

to 

28% 

2. 

EMCE  S 

4  out  of  8 

-26% 

to 

21% 

3. 

SoftCOst-Ada 

2  out  of  7 

-  6% 

to 

5% 

4. 

SYSTEM-3 

2  out  of  8 

-23% 

to 

5% 

5. 

COSTMODL 

1  out  of  6 

-26% 

SASET 

naninal  schedules  were  not 

available. 
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TABLE  3-11 

NCMCNAL  EFPORT  FK1IECITC»IS 


Number 

Ac±ual 

Effort 

SoftCbst- 

Ada 

OOSTMODL 

PRICE  S 

SASET 

SYSTEM-3 

SPQIV20 

■oi 

302 

344 

229 

621 

463 

388 

219 

2 

684 

439 

511 

626 

635 

715 

334 

3 

134 

42 

Not  Run 

36 

102 

70 

219 

B 

692 

Not  Run 

Not  Run 

2604 

2652 

6079 

2562 

5 

144 

517 

438 

749 

814 

761 

286 

6 

190 

109 

94 

529 

394 

131 

100 

7 

696 

508 

308 

1329 

760 

576 

1069 

8 

322 

92 

71 

276 

415 

107 

78 

TABLE  3-12 

RELATIVE  ERRORS  FOR  NOfLINAL  EFFORTS  WHEN  COMPARED  TO  ACTUAL  EFFORTS 


Number 

Actual 

Effort 

SoftCost- 

Ada 

PRICE  S 

SASET 

SYSTEM-3 

SPQIV20 

1 

302 

14% 

-24% 

106% 

53% 

28% 

-27% 

2 

684 

-36% 

-25% 

-  8% 

-  7% 

5% 

-51% 

3 

134 

-69% 

Not  Run 

-73% 

-24% 

-48% 

63% 

4 

692 

Not  Run 

Not  Run 

276% 

283% 

778% 

270% 

5 

144 

259% 

204% 

420% 

465% 

428% 

99% 

6 

190 

-43% 

-51% 

178% 

107% 

-31% 

-47% 

7 

696 

-27% 

-56% 

9]% 

9% 

-17% 

54% 

8 

322 

-71% 

-78% 

-14% 

29% 

-67% 

-76% 
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TABI£  3-13 

NCWINAL  SCHEDUIE  PRQJECTIWS 


Actual 

Schedule 

SoftCost- 

Ada 

OOSTMODL 

PRICE  S 

SYSTEM-3 

SPQR/20 

27 

28.39 

20.0 

36.0 

20.9 

33.3 

38 

35.81 

26.3 

46.0 

22.1 

42.7 

35 

10.34 

Not  Run 

26.0 

11.8 

33.2 

49 

Not  Run 

Not  Run 

50.0 

51.6 

62.7 

12 

33.28 

25.0 

36.0 

26.1 

38.7 

40 

18.14 

14.6 

54.0 

14.6 

30.8 

62 

33.09 

21.6 

53.0 

23.8 

73.6 

56 

14.1 

13.4 

37.0 

13.4 

31.9 

TABLE  3-14 

RELATIVE  ERRORS  FOR  NOtCQ'lAL  SCHEDULES  WHEN  CC^!PARED  TO  ACTUAL  SCHEDULES 


Proj . 
Number 


-26% 


-31% 


Not  Ftin 


33% 

-23% 

23% 

21% 

-42% 

12% 

-26% 

-66% 

-  5% 

2% 

5% 

28% 

200% 

118% 

223% 

35% 

-64% 

-23% 

-15% 

-62% 

19% 

-34% 

-76% 

-43% 

3 . 4 . 3 . 2  Overall  Consistency 


Ndtiiricil  results  were  also  checked  for  consistency.  Table  3-15  am 
3-16  illustrate  the  relative  errors  after  the  calculated  means  were 
applied.  The  number  of  effort  estimates  accurate  within  ±30%  were  as 
follows; 


•  ,  MODEL 

PERPOFMANCE 

RANGE 

1. 

COSTMODL 

3  out  of  6 

-23%  to 

30% 

2. 

SoftCost-Ada 

3  out  of  7 

0%  to 

28% 

3. 

SASET 

3  out  of  8 

-24%  to 

7% 

4. 

SYSTEM-3 

3  out  of  8 

-26%  to 

13% 

5. 

SPQIV20 

1  out  of  8 

-14% 

6. 

FRIGE  S 

1  out  of  8 

-29% 

After  application  of  the 

means,  the  number 

of  schedule  estimates 

accurate  within  ±30%  were  as 

follows: 

MODEL 

PERFORMANCE 

RANGE 

1.  SPQR/20 

6  out  of  8 

-28%  to  20% 

2.  PRICE  S 

5  out  of  8 

-28%  to  29% 

3 .  SYSTEM-3 

3  out  of  8 

-25%  to  19% 

4.  OOSTMODL 

2  cut  of  6 

-27%  to  -23% 

5 .  SoftCost-Ada 

2  out  of  7 

0%  to  14% 

*  SASET  Ncminal  Schedules  were  not  available  for  evaluation. 

3. 4. 3. 3  Analysis  By  Application  Type  and  Contract  Type 

Govemnart  Versus  Ccnmerrrial  Contracts.  Examination  of  ncminal 
performances  identified  the  following  trends: 
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TABI£  3-15 


RELATIVE  ERRORS  FOR  NCMINAL  EFFORT  AFTER  APPLYING  MEANS 


FRCOECT 

NUMBER 

SoftCost-Ada 

Mean=1.75 

COSTMODL 

Mean=1.74 

PRICE  S 
Mean=0.78 

SASET 

Mean=0.70 

SYSTEM-3 

Mean=1.08 

SPQIV20 

Mean=1.18 

1 

99% 

32% 

60% 

7% 

39% 

-  14% 

2 

12% 

30% 

-  29% 

-  35% 

13% 

-  42% 

3 

-  45% 

Not  Run 

-  79% 

-  47% 

-  44% 

93% 

4 

Not  Run 

Not  Run 

194% 

168% 

849% 

337% 

5 

528% 

429% 

306% 

296% 

471% 

134% 

6 

0% 

-  14% 

117% 

45% 

-  26% 

-  38% 

7 

28% 

-  23% 

49% 

-  24% 

-  11% 

81% 

8 

50% 

-  62% 

-  33% 

-  10% 

-  64% 

-  72% 

TABLE  3-16 

RELATIVE  ERROR  FOR  NCMNAL  SCHEDULE  AFTER  APPLYING  MEANS 


roojEcr 

NUMBER 

SoftCost-Ada 

Mean=1.89 

COSTMODL 

Mean=2.1 

PRICE  S 
Mean=0.97 

SYSTEM-3 

MecirF2.05 

SPQIV'20 

Mean=0.94 

1 

99% 

56% 

29% 

59% 

16% 

2 

78% 

45% 

17% 

19% 

6% 

3 

-  44% 

Not  Run 

-  28% 

-  31% 

-  11% 

4 

Not  Run 

Not  Run 

-  1% 

116% 

20% 

5 

424% 

338% 

191% 

346% 

203% 

6 

-  14% 

-  23% 

31% 

-  25% 

-  28% 

7 

0% 

-  27% 

-  17% 

-  21% 

12% 

8 

-  52% 

-  50% 

-  36% 

-  51% 

-  46% 
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Kxlel  ikxxtracy.  SASET  provided  the  most  acxxirate  results  for 
government  contracts  v^tjere  three  of  four  estimates  were  within  28% 
of  the  actual  effort.  The  number  of  effort  estimates  accurate 
ivith-in  ±30%  were  as  follows: 


GOVERNMENT  CX^TTRACTS 


MODEL 

PERFORMANCE 

RANGE 

1. 

SASET 

3  out  of  4 

-  7%  to  29% 

2. 

roicE  S 

2  out  of  4 

-14%  to  -8% 

3. 

SYSTEM-3 

2  out  of  4 

-17%  to  5% 

4. 

(OOSMODL 

1  out  of  4 

-25% 

5. 

SoftCost-Ada 

1  out  of  4 

-27% 

6. 

SPQiV20 

0  out  of  4 

Not  Applicable 

Model  Orcistency.  SofCost-Ada,  Ada  OOCCMD,  and  SYSTEM-3  all 
provided  three  of  four  estimates  to  within  30%  of  the  actucil 
effort  expended.  The  number  of  effort  estimates  accurate  within 
±30%  af  jr  calculated  means  were  aj^lied  were  as  follows: 

MODEL  PERFORMANCE  RANGE 


1. 

SoftCost-Ada 

3 

out 

of 

4 

0%  to  28% 

2. 

SYSTEM-3 

3 

out 

of 

4 

-26%  to  13% 

3. 

C30STM3DL 

3 

out 

of 

4 

-23%  to  30% 

4. 

SASET 

2 

out 

of 

4 

-24%  to  -10% 

5. 

PRICE  S 

1 

out 

of 

4 

-29% 

6. 

SPQR/20 

0 

out 

of 

4 

Not  Applicable 

OCMMERCIAL  OONTOACTS 

Model  Accuracy.  All  models  performed  equally  on  commeioicil 
contracts  so  no  trends  could  be  identified.  The  number  of  effort 
estimates  accurate  within  ±30%  were  as  follows: 

RANGE 


-24% 

14% 

-24% 

-27% 

pQS- 

Not  ^Dplicable 


MODEL 

PERPOFMANCE 

1. 

OOSTMODL 

1 

out  of 

2 

2. 

SoftCost-Ada 

1 

out  of 

3 

3. 

SASET 

1 

out  of 

4 

4. 

SPQH^20 

1 

out  of 

4 

5. 

SYSTEM-3 

1 

out  of 

4 

6. 

PRICE  S 

0 

out  of 

4 
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Mbdel  Oonsistaicy.  No  trends  could  be  identified  for  ccramercial 
ccaitracts.  The  number  of  effort  estimates  accurate  within  ±30% 
after  means  were  applied  were  as  follows: 


MODEL 


PERPOBMftNCE 


RANGE 


1.  SASET 

2.  SPQIV'20 

3 .  SoftOost-Ada 

4.  PRICE  S 

5.  OOSTMODL 

6.  SYSTEM-3 


1  out  of  4 
1  out  of  4 
0  out  of  3 
0  out  of  4 
0  out  of  2 
0  out  of  4 


7% 

-14% 

Not  J^licable 
Not  Applicable 
Not  Applicable 
Not  ^plicable 


^^plicattiicn  TVP^*  Nominal  Performances  in  li^t  of  application  type 
were  as  follows: 


OGMMAND  AND  OC^nROL 

Model  Accuracy.  SASET  provided  the  most  accurate  results 
with  three  of  its  four  estimates  within  24%  of  the  actual 
effort.  The  number  of  effort  estimates  accurate  within  ±30% 
were  as  follows: 


MODEL 

PERFORMANCE 

RANGE 

1. 

SASET 

3  out  of  4 

-  7%  to 

29% 

2. 

SYSTEM-3 

3  out  of  4 

-17%  to 

28% 

3. 

(XSTMODL 

2  out  of  4 

-25%  to 

-24% 

4. 

PRICE  S 

2  out  of  4 

-14%  to 

-  8% 

5. 

SoftCOst-Ada 

2  out  of  4 

-27-6  to 

14% 

6. 

SPQiV20 

1  out  of  4 

-27% 

M^del  Consistency. 

SASET  provided  the  most  accurate  results  with 

three  of  four  esstimates  accurate  to 

within  24% 

of  the  actual 

effort.  The  number  of  effort  estimates 

accurate  within  ±30%  after 

means  were  applied 

were  as  follows: 

MODEL 

PERFORMANCE 

RANGE 

1. 

SASET 

3  out  of  4 

-24%  to 

7% 

2. 

Soft(jost-Ada 

2  out  of  4 

12%  to 

28% 

3. 

SYSTEM-3 

2  out  of  4 

-11%  to 

13% 

4. 

CJOSTMODL 

2  out  of  4 

-23%  to 

30% 

5. 

SPQP/20 

1  out  of  4 

-14% 

6. 

PRICE  S 

1  out  of  4 

-29% 

Mcsdel  Acxairacy.  All  models  performed  equally  on  these  types  of 
systems  in  the  ncminal  runs.  The  number  of  effort  estimates 
accurate  within  +30%  were  as  follows: 


MODEL  PEBFOmmCE  RANGE 


1. 

SASET 

1  out  of  3 

-24% 

2. 

SoftCost-Ada 

0  out  of  2 

Not  Applicable 

3. 

OOSTMDDL 

0  out  of  1 

Not  ^iplicable 

4. 

FRIGE  S 

0  out  of  3 

Not  ^:plicable 

5. 

SYSTEM-3 

0  out  of  3 

Not  Applicable 

6. 

SPQR/20 

0  out  of  3 

Not  Applicable 

Model  C3cnslstenoy. 

All  models  also  performed  equally.  The 

of 

effort  estimates  accurate  within 

+30%  after  calculated 

were  applied  were 

as  follows: 

MODEL 

PERFORMANCE 

RANGE 

1. 

OOSTMODL 

0  out  of  1 

Not  Applicable 

2. 

SoftCost-Ada 

0  out  of  2 

Not  A^licable 

3. 

PRICE-S 

0  out  of  3 

Not  ^^licable 

4. 

SASET 

0  out  of  3 

Not  Applicable 

5. 

SYSTEM-3 

0  out  of  3 

Not  A^licable 

6. 

SPQR/20 

0  out  of  3 

Not  A^licable 

This  page  is  intentionally  left  blank. 


4.0  DATA  ANALYSIS 


4.1  BACKSROUND 


Once  data  was  validated  and  nornalized,  six  projec±s  targeted  in 
the  study  were  statisticcilly  analyzed  to  determine  how  they  ccsmpeired  to 
the  normal  productivity  ranges  eudiibited  by  other  Ada  projects  within 
an  application  donain.  Then,  cost  drivers  inherent  in  the  data  itself 
were  investigated  via  statistical  means  (i.e.,  hypothesis  testing, 
goodness  of  fit,  etc.)  to  validate  cost  drivers  previously  identified 
and  to  uncover  emy  new  cost  drivers. 

A  report  prepared  for  IITRI,  entitled  "Ada  Data  Analysis  and 
Normalization”  (RCI-TR-065) ,  dated  9  January  1989,  contains  results  of 
the  data  analysis  task  that  was  performed  by  Reifer  Consultants,  Inc. 
(PCI) .  Ihe  report  identifies  primary  cost  drivers  for  10  application 
areas  categorized  as  follows; 


1. 

Autcmation 

6. 

Scientific 

2. 

Command  eind  Control 

7. 

Simulation 

3. 

Data  Processing 

8. 

Telecommunications 

4. 

Environment 

9. 

Test 

5. 

Military  (Eii4)edded) 

10. 

Other 

Statistical  sensitivity  analysis  was  conducted  on  the  dcmain  of 
primary  cost  drivers  to  see  if  the  new  Ada  project  data  fit  existing 
relationships  developed  previously  by  PCI.  Data  that  did  not  fit  would 
have  an  enlarged  variance  v^en  ccstpared  to  the  norms  in  each  of  the 
domains.  Hypothesis  testing  was  conducted  to  identify  new  cost 
drivers.  The  result  of  the  analyses  was  an  i^xJated  list  of  cost 
drivers  in  each  of  the  10  application  areas. 
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4.2  RESUIITS 

Domain  Analysis 

The  new  data  provided  by  the  six  projects  fell  within  the  normal 
ranges  for  those  projects  already  within  the  RCI  Ma  database  for  the 
application  domain  noted  in  Table  4-1. 


T^BEE  4-1 

RESDllT  GF  EOfiON  AMAIESIS  [BEIF89] 


Type  of  System  Productivity 

Productivity 

Vaxiance 

Project 

(Dcmain) 

SllX/pn 

Range  (mean) 

(sirxypn) 

1 

Automation 

173 

112-345  (210) 

±  33 

2 

Command  &  Control 

190 

26-289  (180) 

+  30 

3 

Environmental/Tool 

113 

81-587  (240) 

+  26 

4 

Environmental/Tool 

400 

81-587  (240) 

±  26 

5 

Environmental/Tool 

450 

81-587  (240) 

±  26 

6 

Military  (Aixtoome) 

127 

18-228  (143) 

±  18 

Sensitivity  Analysis 

Cost  driver  information  provided  from  the  questionnaires  plus  10 
other  projects  acquired  from  s^jarate  sources  were  used  to  perform  a 
sensitivity  aralysis.  The  new  data  suggested  several  alterations  to 
existing  peurameter  cad  ibrat ions  and  relationships  derived  from  previous 
data.  A  summary  of  primary  cost  drivers  in  PCI's  Ada  database  is 
presented  in  Table  4-2  [REIF89]. 
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TABEE  4-2 


FRIMASY  COBF  ERIVEBS  IN  Rd'S  ADA  DA3ABASE  [REIF89] 


Type  of  Software 

Number  of 
Projects 

Size 

Range 

(ASI£)C) 

Prinary  Cost  Drivers 

Automation 

8 

5-120 

-  Requirements  Volatility 

-  Reuse 

-  Experience 

Conmand  &  Control 

12 

25-1800 

-  Requirements  Volatility 

-  Reuse 

-  Degree  of  Standardization 

-  Experience 

-  Architecture 

Data  Processing 

i 

10 

25-450 

-  Requirements  Volatility 

-  Reuse 

-  Ejqjerience 

-  Architecture 

Environment/Tools 

22 

5-1000 

-  Requirements  Volatility 

-  Reuse 

-  Experience 

Military  (Embedded) 

12 

i  2-250 

-  Requirements  Volatility 

-  Corrplexity 

-  Reuse 

-  Degree  of  Standardization 

-  Degree  of  Real-Time 

-  Experience 

Scientific 

4 

28-300 

-  Requirements  Volatility 

-  Conplexity 

-  Reuse 

-  Experience 

Simulation 

3 

80-175 

-  Requirements  Volatility 

-  Reuse 

-  Experience 

-3 


TABLE  4-2 


PRIMARY  COST  DRIVERS  IN  RCI'S  ADA  DATABASE  (Continued) 


Type  of  Softweire 

Number  of 
Projects 

Size 

Range 

Primary  Cost  Drivers 

Telecxaranunications 

12 

5-400 

-  Requirements  Volatility 

-  Carplexity 

-  Reuse 

-  Experience 

-  Architecture 

Test 

3 

22-125 

-  Requirements  Volatility 

-  Reuse 

-  Experience 

Other 

7 

5-180 

-  Requirements  Volatility 

-  Reuse 

-  Esqjerience 

5.0  OCNCUSICIG 


The  following  six  cost  estimation  models  were  applied  to  a 
database  of  ei^t  caipleted  Ada  projects: 


Ada-Specific  Models 

1.  Ada  oxxayD  (IOC) 

2.  SoftCost-Ada  (1.3) 


Non-Ada  Specific  Models 

1.  PRICE-S  (188230) 

2.  SASET  (1.5) 

3.  SPQIV20  (1.2) 

4.  SYSTEM-3  (j..03) 


The  version  of  each  model  reviewed  is  indicated  in  pjarenthesis  beside 
the  model  name.  Project  questionnaires  were  conpleted  by  the 
developers.  This  information  was  used  to  derive  irputs  for  eaoh  model. 
Eitphasis  was  placed  on  providing  a  consistent  set  of  inputs  across  all 
models.  In  addition,  models  were  applied  using  ncmincd  (average) 
values  for  irput  ratings  viiile  providing  actuad  project  values  for 
size,  application  type,  programming  language,  and  other  model  irputs 
that  must  be  estimated  early  in  the  life  cycle  and  for  vhich  there  is 
no  associated  average  value.  The  nominal  inputs  reflect  the  level  of 
knowledge  about  a  new  project  prior  to  contract  award.  Resultant 
analyses  were  based  on  a  cojtparison  of  each  model's  schedule  and  effort 
projection  and  nominal  run  r^  'Its  to  the  actual  effort  expended  by  the 
software  develc^jer. 

Results  were  evaluated  for  accuracy  and  consistency  for  each  of 
the  following  six  categories: 

1.  Overall  effort 

2 .  Overall  schedule 

3 .  Government  contracts 

4 .  Commercicil  contracts 

5.  Command  and  control  applications 

6.  Tools  and  environment  afplications. 
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Tables  5-1  and  5-2  respectively  summarize  the  test  case  study  results 
for  derived  and  ncndnal  inputs.  For  each  evaluation  criteria,  the  two 
models  that  had  the  hipest  performance  ratings  are  listed.  Model 
results  were  also  evaluated  based  on  the  project's  design  approach  and 
personnel  experience  with  Ada.  Model  performeinces  could  not  be 
correlated  to  the  language  considerations  of  the  models  which  are 
described  in  Section  2. 


5.1  REOaiMENDATICWS 

The  test  case  study  results  demonstrate  the  benefits  of  cost 
models  that  cissist  the  estimator  in  predicting  resource  requirements 
for  a  new  develcpnent.  The  results  do  not  vcilidate  the  need  for  Ada- 
specific  models.  Althoui^  SoftCost-Ada  was  most  accurate  overall,  non- 
Ada-specific  models  were  ccmparable  in  terms  of  accuracy  and 
consistency. 


The  tesiiLts  do  reccanmend  that  users  consider  the  following  to 
determine  whicii  models  should  be  applied  to  estimate  Ada  software 
costs: 


Assess  how  nuch  informaticn  is  a[vailable  about  the  project 
and  the  developing  organization.  ^plication  of  models  to 
both  regular  and  nominal  runs  in  the  study  shewed  that  model 
performances  varied  with  differing  amount  of  project 
information.  Scare  performed  better  with  minimum  information 
while  others  performed  better  with  detailed  information. 

Oonsider  the  customer.  Mcdel  performances  were  evaluated 
betsed  on  the  type  of  contract.  Scare  models  were  more 
effective  when  applied  to  govemnent  contracts  vhile  others 
were  more  accurate  for  estimating  ccannercial  contracts. 

Oensider  the  type  of  application.  Projects  targeted  in  the 
test  case  study  consisted  of  three  different  types  of 
applications:  command  and  control  (4  projects), 
tools/environnvent  (3  projects),  ard  avionics  (1  project).  An 
cmedysis  of  results  based  on  application  typje  revealed  that 
models  that  were  most  accurate  on  ccanmand  and  control 
applications  were  not  eis  accurate  for  tools  and  environment 
applications. 
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TABLE  5-1 


I 
I 

I  SUWffiRY  TEST  CASE  STUDY  RESULTS: 

BEST  TWO  PERFORMANCES  IN  EACH  CATEGORY 

I  _ 


Evaltiatlcn 

Criteria 

Model 

Performance 
(Within  30%) 

Range 

Overall  Accuracy 
of  Effort 

SoftCost-Ada 

SASET 

4  out  of  7 

4  out  of  8 

0%  to  13% 
-29%  to  -29% 

Overall  Accuracy 
of  Schedule 

SYSTEM-3 

PRICE  S 

4  out  of  8 

3  out  of  8 

-27%  to  -  7% 
3%  to  18% 

Overall  Consistency 
of  Effort 

SYSTEM-3 

PRICE  S 

5  out  of  8 

5  out  of  8 

-14%  to  28% 
-26%  to  22% 

Overall  Consistency 
of  Schedule 

SYSTEM-3 

IRICE  S 

5  out  of  8 

5  out  of  8 

0%  to  28% 
-29%  to  28% 

Model  Accuracy  on 
Government  Contracts 

SASET 

OOSTMODL 

3  out  of  4 

2  out  of  4 

-  7%  to  29% 
-25%  to  -  1% 

Model  Consistency  on 
Government  Contracts 

SYSTEM-3 

PRICE  S 

4  out  of  4 

3  out  of  4 

-14%  to  28% 
-26%  to  0% 

Model  Accuracy  on 
Commercicil  Cdntracts 

SoftCost-Ada 

SPQR/20 

3  out  of  3 

1  out  of  4 

0%  to  6% 
-22% 

Model  Consistency  on 
Cornmercicil  Contracts 

SoftCost-Ada 

PRICE-S 

3  out  of  3 

2  out  of  4 

-13%  to  -  8% 

-  1%  to  22% 

Model  Accuracy  on 
Ccnimand  &  Control 
Applications 

SASET 

SPQIV20 

3  out  of  4 

3  out  of  4 

-  7%  to  29% 
-22%  to  19% 

Model  Consistency  on 
ccanmand  &  Control 
^plications 

PRICE  S 

SASET 

4  out  of  4 

3  out  of  4 

-26%  to  0% 
-15%  to  1% 

Model  Accuracy  on 
Tools/Environment 
Applications 

SoftCost-Ada 

SASET 

2  out  of  2 

1  out  of  3 

0%  to  2% 
-29% 

Model  Consistency  on 

Tools/Environment 

Applications 

SoftCost-Ada 

SYSTEM-3 

2  out  of  2 

1  out  of  3 

-13%  to  -11% 
28% 
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TABLE  5-2 

SUMMARY  TEST  CASE  STUDY  RESUITS  FOR  NCMINAL  RUNS: 
BEST  TWO  PERPOFMANCES  IN  EACH  CATEGCRY 


Evaluation 

Criteria 

Model 

ferfooanoR 
(Within  30%) 

Range 

Overall  Accuracy 
of  Effort 

SASET 

SYSTEM-3 

4  cut  of  8 

3  cut  of  8 

-24%  to  29% 
-17%  to  28% 

Overall  Accuracy 
of  Schedule 

SPQIV20 

PRICE  S 

6  out  of  8 

4  cut  of  8 

-23%  to  28% 
-26%  to  21% 

Overall  Consistency 
of  Effort 

GOSTMDDL 

SoftOost-Ada 

3  out  of  6 

3  out  of  7 

-23%  to  30% 

0%  to  28% 

Overall  Consistency 
of  Schedule 

SPQFV'20 

ERICE  S 

6  cut  of  8 

5  cut  of  8 

-28%  to  20% 
-28%  to  29% 

Model  Accuracy  on 
Government  Contracts 

SASET 

PRICE  S 

3  cut  of  4 

2  cut  of  4 

-  7%  to  29% 
-14%  to  -  8% 

Model  consistency  on 
Government  Contracts 

SoftCost-Ada 

SYSTEM-3 

3  out  of  4 

3  out  of  4 

0%  to  28% 
-26%  to  13% 

Model  Accuracy  on 
Conmercial  Contracts 

COSTMDDL 

SoftCost-Ada 

1  out  of  2 

1  out  of  3 

-24% 

14% 

Model  Consistency  on 
Conimercial  Contracts 

SASET 

SPQEV20 

1  out  of  4 

1  out  of  4 

7% 

-14% 

Model  Accuracy  on 
Ccjnmand  &  Control 
^plications 

SASET 

SYSTEM-3 

3  cut  of  4 

3  out  of  4 

-  7%  to  29% 
-17%  to  28% 

Model  Consistency  on 
Command  &  Control 
Applications 

SASET 

SoftCost-Ada 

3  out  of  4 

2  out  of  4 

-24%  to  7% 
-12%  to  28% 

Model  Accuracy  on 
Tools/Environment 
Applicaticxis 

SASET 

1  cut  of  3 

-24% 

Model  Consistency  on 

Tools/Environment 

Applications 

0 
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5.2  TrARNRD 


While  the  stucfy  was  performed  on  six  cost  estimation  models,  there 
are  many  other  models  that  are  currently  being  used  to  estimate  Ada 
software  costs.  Additionally,  as  Ada  usage  and  technology  increase, 
model  developers  will  be  constantly  refining  their  models  to  provide 
more  accurate  results  in  this  field.  Two  models  reviewed  in  this 
study,  SoftCost-Ada  and  SYSTEM-3  (now  SYSTEM-4) ,  have  since  released 
new  versions  of  their  models.  Ada  aX3CM0  is  an  initial  operational 
capability  that  will  be  subject  to  further  refinement.  Because  of 
these  factors,  it  is  iirportant  to  identify  lessons  learned  during  the 
test  case  study  to  facilitate  further  reseeirdi  in  this  area. 


Lesson  NO.  1:  Have  a  specific  objective  in  mind 
vhen  you  afproach  a  develc^jer  for  data;  Use  a 
sinple,  concise  data  collection  form  that  focuses 
only  on  the  collection  c^j active. 


Many  requests  were  made  to  individuals  to  provide  data  on  a 
voluntary  basis  for  the  test  case  study.  A  specific  objective  provided 
an  incentive  to  the  develcper  to  provide  data,  especially  if  the 
develcper  knew  vhat  the  data  was  being  used  for  and  that  results  of  the 
data  analysis  would  be  valuable  to  his  organization.  Ihe  test  case 
stxjdy  results  were  of  great  interest  to  all  of  the  developers  v*io 
provided  data  for  the  research. 

A  single  cSajective  also  focuses  collection  activities  for  the 
developing  organization.  Developers  cure  more  likely  to  provide  data  if 
it  won't  require  too  much  effort.  Requesting  a  broad  range  of 
information  frcm  a  single  orgcinization  (i.e.,  cost  data,  quality  data, 
productivity  data)  will  likely  result  in  losing  a  potential  data 
resource. 


Ada  project  data  was  provided  for  each  project  in  the  form  of  a 
ccmpleted  project  questionnaire.  Initially,  Software  Project  Data 
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Collection  Forms  and  Instructions  occpiled  for  the  CcEptroller's  Office 
(ESD/AOCR)  were  sent  to  developers.  These  forms  were  tailored 
specifically  to  the  Ada  language  and  included  many  of  the  data  items 
required  to  run  sane  of  the  major  softwcure  cost  cind  schedule  estimating 
models.  Three  projects  targeted  in  the  test  case  stu<^,  attained  from 
ESD,  were  already  in  this  format.  However,  the  feedback  from  project 
developers  regeu^ding  the  volume  of  information  requested  caused  IITRI 
to  develop  new,  shorter  forms  devised  from  an  examination  of  model 
irput  peurameters. 


It  is  also  necessary  to  cite  the  minimum  data  that  is  needed 
before  the  develcper  uses  valuable  time  and  effort  to  complete  the 
forms.  For  exaitple,  at  minimum,  the  study  required  project  effort  in 
person  months  and  project  size  in  SLOC.  The  "We'll  take  anything 
you've  qot"  attitude  results  in  too  little  data  that  is  too  general  to 
fulfill  any  useful  purpose. 

No  matter  hew  simple  or  well  defined  a  form  is,  raw  data  provided 
by  the  develcper  needs  to  be  validated.  Through  validation  procedures, 
inconsistencies  between  responses  can  be  identified.  It  is  cdso  a  good 
idea  to  discuss  the  completed  questionnaire  with  the  person  who 
completed  the  form  and  "spot-check"  certain  questions  to  ensure  that 
the  develcper  had  the  cx>rrect  context  in  mind. 


Lesson  No.  2:  Expec±  a  one  in  five  response  from 
developers  vho  initially  agree  to  provide  project 
eJata. 


inejuiries  into  the  projects  listed  in  the  AJPO  Ada  Usage  Databeise 
resulted  in  19  respondents  vho  agreed  to  provide  data  for  the  test  case 
stuidy.  Of  these  newly  identified  projects,  four  were  eventually  able 
to  provide  reejuested  eJata  in  a  timely  manner.  Reasons  for  not 
providing  ciata  included  unavailability  of  information  reiguested,  the 
amount  of  effort  that  conpleting  the  form  would  entail,  and  nonapproval 
by  projec±  offices  to  release  the  data. 
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Fran  this,  we  have  learned  the  importance  of  targeting  a  large 
number  of  organizations  for  a  ^lecific  collection  cbjective.  imil's 
status  as  a  not-for-profit  research  institute  and  non-ccnpetitor  also 
enhanced  our  position  to  acquire  data. 

lesson  No.  3:  Provide  an  incentive  for  the 
developer  to  provide  data. 

Projects  targeted  in  the  test  case  study  were  obtained  with  the 
incentive  that  the  results  of  the  test  case  study  would  be 
aubonatically  provided  at  no  cost  to  the  developer.  In  specific  cases 
other  incentives  were  utilized.  For  example,  one  developer  did  an 
excellent  job  of  ccnpleting  the  project  questionnaire  but  omitted  the 
number  of  person  months  required  to  carplete  the  effort.  The  project 
data  that  was  provided  by  the  develcper  could  not  fulfill  its  intended 
purpose  unless  the  actual  project  effort  was  also  included  in  the  data, 
l^n  contacting  the  developer,  we  were  told  that  the  information  was 
proprietary.  Because  the  project  was  funded  by  the  government,  the 
information  was  eventually  obtained  through  government  intervention. 

In  all  cases,  the  ability  to  gain  the  cooperation  of  software 
developers  required  diligence  cm  follcwing-ip  initial  requests  and  on 
an  agreeable  attitude. 

Lessen  No.  4:  Identify  what  needs  to  be  collected 
at  the  beginning  of  a  new  project  and  have  the 
developer  provide  the  information  at  the  project's 
coipletion. 

It  is  interesting  to  note  that  six  of  the  ei<^t  projects  eveiluated 
for  the  study  indicated  extremely  tight  schedule  constraints.  In  view 
of  these  accelerated  schedules,  data  acquisition  cannot  be  considered 
to  be  a  constant  nuiseince  to  the  developer.  However,  the  developer 
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needs  to  be  avrare  of  the  information  that  needs  to  be  collected  so  that 
it  can  be  traciked  as  the  project  progresses.  Information  may  be  too 
difficult  to  retrieve  at  the  point  of  the  project's  ccnpletion.  For 
exaitple,  the  Air  Force  wanted  HIKE  to  eveduate  the  distribution  of 
project  effort  by  life-cycle  phase.  Hcwever,  none  of  the  developers 
\Aio  provided  data  for  the  costing  study  tracked  effort  ejqjended  by 
phase.  It  would  have  been  beneficial  to  the  costing  study  to  identify 
data  items  at  the  beginning  of  each  project,  such  as  in  the  Request  for 
the  Proposal  (RFP) . 

lessen  Nb.  5:  Ihe  best  way  to  gather  consistent 
information  to  provide  collection  tools. 

Ihe  Ada  costing  study  enphasized  the  different  conventions  that 
various  devel<^)ers  use  to  measure  software  attributes.  Developers  who 
provided  data  used  one  of  three  methods  to  count  SDDC: 

1.  Physical  lines  including  cemments  and  blank  lines 

2.  Physical  lines  excluding  cemments  and  blank  lines 

3.  Terminal  semicolons  including  those  used  in  data  declarations 
and  formal  parameter  lists. 

In  order  to  normalize  the  line  counts  so  that  conparable  information 
would  be  evaluated  in  the  data  analj^is,  conversion  factors  were  used 
to  standardize  the  line  counts.  For  exairple,  deciding  that  an  Ada 
source  line  of  code  should  be  defined  using  terminal  semicolons, 
physical  line  counts  (excluding  comments  and  blank  lines)  were  reduced 
by  20%  for  a  comparable  measure  of  size.  (S^viously,  the  need  to  derive 
conversion  factors  would  not  have  been  required  if  an  autcroated  line¬ 
counting  tool  had  been  provided  to  each  developer. 

Lesson  No.  6:  For  validation  purposes,  it  is  best 
to  obtain  project  data  directly  from  the  developer 
rather  than  from  the  government  contact. 


It  is  best  to  cxiiinunicate  directly  with  the  developer,  for  it  is 
the  developer  vho  can  answer  questions  about  the  nature  of  the 
develcpment.  It  is  also  iirportant  that  personnel  at  the  prcper  level 
be  designated  to  respond.  To  illustrate  this  point,  ITTRI  had  occasion 
to  ask  three  pecple  on  the  same  project  to  rate  the  capability  of  the 
programming  staff:  the  section  manager,  the  project  manager,  and  one  of 
the  programming  staff.  Each  respondent  had  a  different  perspective  on 
the  development  and  each  relied  with  a  different  evaluation.  It  is 
important  to  be  awctre  of  these  tmique  perspectives  and  when  feasible, 
use  consistency  vhen  targeting  project  personnel  for  data  collection. 


- 1 

lessen  No.  7:  Designate  a  single  collector  and 
foc2d  point  for  all  interface  with  the  developer. 


One  method  that  will  result  in  more  accurate  data  is  to  designate 
a  single  focal  point  for  interface  with  the  software  developer.  This 
approach  enables  the  acquisitioner  to  beceme  familiar  with  the  project 
and  be  able  to  spot  inconsistencies  in  the  project  data.  Because  the 
acquisitioner  is  more  familiar  with  the  develcproent,  there  is  a 
tendency  to  cisk  less  questions  required  to  familieurize  oneself  to  a 
development.  A  single  interface  also  allows  a  good  rapport  to  develop 
between  developer  and  data  acquisitioner.  After  assessing  these 
factors,  models  which  meet  the  given  criteria  for  the  chosen  project 
should  be  ajplied  to  generate  cost  estimates. 

5.3  TOPICS  FOR  FURTHER  RESEAIXH 

This  report  represents  a  snail  window  in  time  with  respect  to  the 
state  of  the  art  in  software  cost  estinating.  case  studies  are 
beneficial  because  they  provide  definitive  data  that  reflect  on  current 
practices.  As  long  as  Ada  development  methodologies  continue  to 
evolve,  new  case  studies  are  warranted.  We  recemmend  that  future  case 
studies  consider  the  follcwing; 
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o  Assessment  of  Ada  issues:  An  evaluation  of  projecrt  data  in 
lic^t  of  Ada  issues  described  in  Section  2.2  shewed  that  six 
of  the  ei<^t  issues  could  not  be  evaluated  due  to  a  lack  of 
data.  For  exanple,  the  distribution  of  effort  by  life-cycle 
phase  could  not  be  evaluated  because  none  of  the  develc^jers 
v4io  provided  data  for  this  study  tracked  effort  expended  by 
phase.  Schedule  and  effort  distribution  for  Ada  versus  non- 
Ada  projects  is  a  significant  Ada  issue.  Future  case  studies 
should  consider  these  issues  and  target  the  data  that  is 
needed  to  evaluate  them. 

o  Inclusion  of  other  models:  Models  that  were  included  in  this 
study  were  selected  based  upon  their  availability  to  either 
the  AFCSTC,  USACEAC,  or  IITRI.  Model  vendors  were  not 
solicited.  There  are  hewever  a  number  of  other  good  models, 
for  example,  SUM  and  SEER,  that  reseeuodiers  should  consider 
evciluating  in  future  case  studies. 

o  Data  acquisition  that  targets  specific  application  types: 
Findings  indicate  seme  interesting  trends  based  on  the  type 
of  contract  -  cxaranercicil  or  government  -  and  the  project's 
application  type.  Future  studies  should  consider  these 
trends  and  target  projects  based  on  these  evaluation 
criteria.  For  example,  it  would  be  beneficial  to  evaluate 
model  performance  for  business  applications. 

o  Maintananoe  cost  evaluation:  It  is  generally  perceived  that 
Ada  usage  will  result  in  long-term  benefits  of  reduced  costs 
required  to  maintain  code  that  is  less  error  prone.  Cost 
models  usually  provide  life-cycle  mainteneuxo  cost  for  a 
specified  term.  It  would  be  useful  to  conpare  predicted 
costs  to  actucil  Ada  maintenance  experience. 
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APEBOnX  A 


MDCEL  VENDCRS/PODnS  OF  COmCT  (POC) 


Each  of  the  models  included  in  this  study  are  undergoing  continual 
revision  as  develcpers  receive  feedback  from  their  users.  For 
additional  information  about  a  model  or  package,  the  designated 
vendor/point  of  contact  listed  in  Table  A-1  should  be  contacted. 


TABLE  A-1 

MDraii  VENDORS/FOINIS  OF  COmACT  (POC) 


MODEL 

VENDOR/POC 

Ada  OOOCMO 

Dr.  Beurry  Boehm 

TRW  Defense  Systems  Group 

One  Space  Park 

Redondo  Beach,  CA  90278 
(213)  812-0786 

FRICE  S 

Dr.  Robert  E.  Park 

PRICE  Systems 

General  Electric  Corrpany 

300  Route  38,  Bldg.  146 

Moorestcwn,  NJ  08057 

1-800-GE-PRICE 

SASET 

Mr.  Steve  Gross 

Naval  Center  for  Cost  Analysis 
Department  of  the  Navy 

Washington,  DC  20350-1100 
(202)  694-0173 

SoftCost-Ada 

Mr.  Doncild  Reifer 

Reifer  Consultants,  Inc. 

25550  Hawthorne  Blvd,  Suite  208 
Torrance,  CA  90505 
(213)  373-8728 

TABIE  A-1  (Continued) 
COST  MDDEL  POINTS  OF  CCmkCT 


SPQIV20 

Mr.  Wayne  Hadlock 

Software  Produc±ivity  Research,  Inc. 

P.O.  Box  1033 

1972  Massachusetts  Avenue 

Cainbridge,  MA  02140 
(617)  495-0120 

SYSTEM-3 

Mr.  Wayne  Stanley 

Ccnputer  Efcoronics,  Inc. 

Suite  109 

4560  A±tdrcLLty  Way 

Marina  del  Rey,  CA  90292-5424 
(213)  827-7300 

DoD;  Lt.  Paul  Meirsey 

Wri^t-Patterson  AFB 
(513)  255-6347 

TABLE  A-2 


AEA  CXXTMD  IMPLEIffiMMTCNS  POINTS  OF  a^TTACT  (POC) 


PACKAGE 

POC 

EMO* 

Lt.  Darrish 

Headqucirbers  EMO-ACS 

Norton  AFB,  CA  92409-6468 

(714)  382-4713  Autovon:  876-5836 

OOSTAR 

Mr.  Den  Ligett 

Softstar  SystCTis 

28  Ponemah  Road 

Amherst,  NH  03031 
(603)  672-0987 

COSTMODL 

Mr.  Bemie  Roush 

NASA  Jchnson  Space  Center 

Mail  Code  EM  7 

Houston,  TX  77058 
(713)  483-9092 

GEOOMO 

Ms.  Susan  Boers 

GEC  Software 

1850  Centennial  Park  Drive,  Suite  300 
Reston,  VA  22091 
(703)  648-1551 

Mr.  Peter  Sizer 

132-135  Long  Acre 

London  WC2E  England 

44-1-240-7171 

*  Currently  does  not  include  Incremental  Development. 
Restricted  use  to  Government  only. 
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afunodc  b 


KXEL  INEUr  PARAMEIBS:  rPTATTm  DEFINITrCKS 


This  appendix  contains  a  detailed  description  of  model  input 
parameters  for  autanatad  models  discussed  in  Section  2.0.  Ihe  input 
parameter  descriptions  cure  listed  by  model  name. 

.  (1)  Ada  OOOCMD 

(2)  PRICE  S 

(3)  SASET 

(4)  SoftCost-Ada 

(5)  SPQR/20 

(6)  SYSTEM-3 

Input  variables  for  each  model  are  categorized  by  input  type  as 
follows: 

•  Product  attributes  describe  the  characteristics  of  the 
software  to  be  developed. 

•  Sizing  factors  determine  the  quantity  of  software  to  be 
developed. 

■  Process  attributes  describe  the  develcpnent  methodology: 

requirements  definition,  use  of  software  tools,  modem 
programming  practices,  schedule,  supporting  documentation, 
etc. 


Computer  attributes  refer  to  the  virtual  machine  underlying 
the  system  to  be  develc^jed. 

Personnel  attributes  address  the  experience  and  capabilities 
of  the  software  developnent  team  assigned  to  the  project. 

Environmental  factors  describe  other  external  circumstances 
that  affected  the  develcpment  such  as  security  requirements, 
the  makeip  of  the  organizations  involved,  development 
locations,  and  office  facilities. 
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The  description  for  each  parameters  contains  the  follcwing 
information; 

•  Name  of  iiput  variable 

•  Type  of  iiput  -  Input  value  formats  are  categorized  as 
follcws: 


R  =  Rating 

%  =  Percentage  or  proportion 

#  =  Count 

C  =  Category 

S  =  Character  string 

Y/N  =  Yes  or  No 

E  =  Errpirically  derived 

•  Default  value 

•  Required  or  optional  input 

■  Input  variable  definition  -  Descriptions  provided  here  cire 
summary  explanations.  More  detailed  definitions  and 
additional  clarification  of  inputs  are  provided  in  the 
training  for  use  of  the  model  or  in  the  model's 
reference/user's  manual. 

The  format  for  each  variable  description  is  as  follows: 

Name  [type,  default,  required  or  optional]  -  input  variable  definition. 
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Ada  GOCXMO 


Prcxiuct  Attributes 

1.  Bequired  software  reliability  [R,  Default  =  naninal, 
Required]  -  degree  of  reliability  required  for  a  softwcire 
subsystem  measured  in  terms  of  the  effect  of  a  software 
failure. 

2.  Software  product  ocrpleacLty  [R,  Default  =  nominal,  Required] 
-  level  of  complexity  of  the  module  to  be  develcped  as  a 
function  of  the  type  of  operations  to  be  performed  by  the 
module:  control,  ccnputation,  device-dependent,  or  data 
management. 

3.  Required  reusability*  [C,  Default  =  Ada  components  not  for 
reuse  elsevrtiere,  Requir^]  -  the  extent  that  the  developer 
must  design,  document,  and  test  Ada  components  for  reuse  in 
another  application. 

Sizing  Factors 

1.  Delivered  source  instructions  [#,  No  default.  Required] - 
count  of  all  program  instructions  created  by  project 
personnel  based  on  the  number  of  carriage  returned  in  package 
specifications  plus  the  number  of  semicolons  in  package 
bodies. 

2.  Adapted  or  reused  instructions  [4,  No  defaiHt,  Requirec.]- 
count  of  delivered  source  instnxrtions  adapted  frcsn  existing 
softwcure  to  form  the  new  product. 

Adapted  Code  Modification  Factors: 

3.  Design  modified  [%,  No  defaxilt.  Required]  -  percentage  of  the 
adapted  software's  design  Vi^ch  is  modified  in  order  to  adapt 
it  to  the  new  project's  <±)jectives  and  environment. 

4.  C3ode  modified  [%,  No  default.  Required]  -  percentage  of  the 
adapted  softwcue's  code  which  is  modified  in  order  to  adapt 
it  to  the  new  project's  objectives  and  environment. 

5.  Integration  required  far  mcxlified  software  [%,  No  default. 
Required]  -  amount  of  effort  required  to  integrate  and  test 
the  adapted  software  into  cui  overall  product  as  compared  to 
the  normal  amount  of  integration  and  test  effort  for  software 
of  comparable  size. 


*  New  cost  driver  for  Ada  OOOCMO 
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6. 


Data  base  size  [R,  Default  =  nominal,  Required]  -  ratio  of 
amount  of  data  to  be  assembled  and  stored  in  main  storage  by 
the  time  of  software  acceptance  to  the  total  program  size  in 
delivered  source  instructicais. 

Process  Attributes 

1.  Experienoe  with  Ada  Process  Model*  [R,  No  default,  Required] 
-rating  of  the  amount  of  familiarity  and  use  of  the  TRW  Ada 
Process  Model. 

2.  Design  thocouc^hness  at  FDR:  Dhit  package  specs  ccnpiled, 

bodies  outlined*  [R,  No  default.  Required]  -  degree  to  vtiich 
ccrpilable  package  specifications  (and  body  outlines)  were 
produced  for  all  tc^level  and  critical  lower-level  Ada 
packages  by  PER. 

3.  Risks  eliminated  by  PER*  [R,  No  default.  Required]  -  degree 
to  v^iich  all  major  risk  items  are  identified  and  elimirated 
by  PER. 

4.  Requirements  volatility  during  development*  [R,  No  default. 

Required]  -  frequency  and  criticality  of  changes  in 

requirements  during  product  development. 

5.  Use  of  modem  progranming  practices  [R,  Defaiilt  =  ncsninal. 
Required]  -  degree  to  which  modem  programming  practices  are 
used  in  developing  the  software. 

6.  Itee  of  software  tools  [R,  Default  =  nominal.  Required] - 

degree  to  which  software  tools  are  used  in  develcping  the 
software  subsystem. 

7.  Required  develcpment  schedule  [R,  Default  =  ncsninal. 
Required]  -  level  of  schedule  constraint  inposed  on  the 
project  team  developing  a  software  subsystem  expressed  in 
terms  of  the  percentage  of  schedule  stretchout  or 
acceleration  with  respect  to  a  notninal  schedule  with  notiineil 
effort. 

CcTOPUter  Attributes 

1.  Execution  time  ocnstraint  [R,  Default  =  noninal,  Required]- 
degree  of  execution  time  constraint  inposed  upon  a  subsystem 
expressed  in  terms  of  the  percentage  of  available  execution 
time  expected  to  be  used  by  the  subsystem  and  any  other 
subsystem  consuming  the  execution  resources. 

*  New  cost  driver  for  Ada  OOC3CMO 
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2.  MEOn  storage  ocnstraint  [R,  Default  =  ncniinal,  Required]- 
degree  of  main  storage  constraint  imposed  on  a  software 
subsystem  ejq)ressed  in  terms  of  the  percentage  of  main 
storage  ei^jected  to  be  used  by  the  subsystem  and  any  other 
subsystems  consuming  the  main  storage  resources. 

3.  Virtual  machine  volatility:  Hcet*  [R,  Default  =  nominal, 
Required]  -  relative  frequency  of  major  changes  and  minor 
changes  to  the  host  virtual  machine. 

4.  Virtual  maciiine  volatility:  Target*  [R,  Default  =  nomincil. 
Required]  -  relative  frequency  of  major  changes  and  mirtor 
changes  to  the  target  virtual  machine. 

5.  Osqpiiter  turnaround  time  [R,  Default  =  ncmined,  Required]- 
average  response  time  in  hours  from  the  time  the  developer 
submits  a  job  to  be  run  until  the  results  sue  back  in  the 
developer's  hands. 

Personnel  Attributes 

1.  Analyst  capability  [R,  Default  =  nominal,  Required]  -  level 

of  capability  of  the  analysts  expressed  in  terms  of 
percentiles  with  respect  to  the  overall  peculation  of 

analysts. 

2.  Programmer  capability  [R,  Default  =  roninal,  Required] - 

level  of  capability  of  the  analysts  expressed  in  terms  of 
percentiles  with  respect  to  the  overall  peculation  of 

analysts. 

3.  Applications  experience  [R,  Default  =  ncmineil,  Required  ]- 
the  project  team's  equivalent  level  of  experience  with  the 
type  of  ajcJlication  to  be  developed. 

4.  Virtual  machine  experience  [R,  Default  =  ncmincd,  Reqviired]- 

the  project  team's  equivalent  level  of  experience  with  the 
underlying  virtual  machine  for  an  application. 

5.  Prograianing  langucige  eoqjerience  [R,  Default  =  nominal, 
Required]  -  the  project  team's  equivalent  level  of  ejq)erience 
with  the  programming  language  to  be  used. 

Environmental  Factors 

1.  Classified  security  applicaticn*  [R,  Default  =  ncminal. 

Required]  -Unclassified  (ncminal  rating)  or  classified  (high) 
project  security  classification. 

*  New  cost  driver  for  Ada  OOCEMD 
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ERECE  S 


Product  Attributes 

1.  Platform  [R,  No  default.  Required]  -  measure  of 
specifications  that  must  be  met  regarding  the  portability, 
reliability,  structuring,  testing,  and  documentation  required 
for  aoc^Jtable  contract  performcince. 

2.  language  [S,  No  default.  Required]  -  source  language  to  be 
used  for  the  software  develcpnent  effort. 

3.  HSI  ocxqplexity  [R,  Default  =1.0,  Required]  -  relative  effect 
of  cotplicating  factors  <xi  the  software  development  task 
caused  by  hardware/softvare  interactions. 

4.  Scuroe  oode  mix  [%,  No  default.  Required]  -  percentage  of 
deliverable  source  lines  of  oode  tliat  performs  each  of  the 
follcwing  categories  of  cperations: 


•  Data  storage  and  retrievad 

•  On-line  ocnmnicatlcns 

•  Real-time  ocnmand  and  ocntrol 

•  Interactive  operaticns 

•  Mathematical  applicaticns 

•  String  manipulaticns 

•  Operating  systems,  and/or 

•  Other. 

5.  User  defined  application  [R,  No  default.  Required  if  a 
percentage  of  source  lines  of  oode  is  allocated  to  the  Other 
category  of  operation]  -  describes  the  instruction  mix  of 
software  corresponding  to  Other  (i.e.  diagnostics,  array 
processing,  quad-redundant  operating  systems,  etc.). 

Sizing  Factors 

1.  Scuroe  lines  of  oode  [#,  No  default.  Required]  -  total  number 
of  source  lines  of  code  to  be  developed  excluding  comments. 

2.  Ntan-exBcutable  oode  [%,  No  default.  Required]  -  fraction  of 
source  lines  of  code  describing  type  declarations,  data 
statements,  data  initializations,  and  instantiations. 

3.  Nev  design  (corresponding  to  source  code  mix)  [%,  No  default, 
Required]  -  amount  of  new  design  required  for  the  amount  of 
software  for  each  category  of  operation  designated  under 

scuroe  oode  mix. 
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4.  New  cadet  (corresponding  to  scurce  code  mix)  [%,  No  defaxilt, 
Required]  -  amount  of  new  coding  required  for  the  amount  of 
software  for  each  category  of  operation  designated  under 
source  code  mix. 

Process  Attributes 

1.  Internal  integpration  [R,  No  default,  Required]  -  level  of 
difficulty  of  integrating  and  testing  softwcure  conponents  to 
the  CSCI  level. 

2.  External  integration  [R,  No  default,  Required  only  if  system 
level  integration  is  to  be  estimated]  -  level  of  difficulty 
of  integrating  and  testing  the  CSCIs  to  the  system  level. 

3.  Schedule  [S,  No  defaiilt.  Either  SCR  or  SSR  dates  required, 
other  dates  cune  optioneLL]  -  Milestone  dates  relative  to  the 
single  CSCI  being  estimated: 

•  System  Cono^jt  effort  starts 

•  System  Requirements  Review 

•  System  Design  Review  (SCR) 

•  Software  Specification  Review  (SSR) 

•  Preliminary  Design  Review 

•  Critical  Design  Review 

•  Test  Readiness  Review 

•  Functional  Configuration  Audit 

•  Physical  Configuration  Audit 

•  Formal  Qualification  Review 

•  Operational  Test  eind  Evaluation. 

PRICE  S  generates  a  schedule  based  on  user-supplied  data 
in  addition  to  an  naniral  or  typical  schedule  based  on 
the  SER  or  SSR  date  that  is  irput. 

Computer  Attributes 

1.  Utilization  [%,  No  default.  Required]  -  fraction  of  available 
harxiware  cycle  time  or  total  memory  capacity  used. 

Environment  Factors 

1.  Ifanagement  ccnplexity  [R,  No  default.  Required]  -  relative 
effect  of  complicating  factors  on  the  softWcure  task  such  cis 
development  at  more  than  one  location  and/or  a  multinational 
project. 

2.  Productivity  factor  [E,  No  default.  Required]  -  net  effect  of 
organizatioral  characteristics  including  such  factors  as 
skill  levels,  experience,  productivity,  and  efficiency. 
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3. 


CcnplescLty  [R,  Default  =  1.0,  Required]  -  relative  effect  of 
ccnplicating  factors  on  the  software  develc^ment  tasks 
including  such  factors  as  product  familiarity,  personnel 
skills,  software  tools,  new  language,  and  changing 
requirements. 
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SASET 


Produc±  Attributes 

1.  Class  of  softuare  [C,  No  default,  Required]  -  operating 
environnent  of  the  software  to  be  developed;  manned  flight, 
unmanned  flight,  avionics,  shiFfcoard/sufcJtarine,  ground, 
ccEJnercicil . 

2.  Primary  Software  language  [C,  No  default.  Required] - 
principle  software  language  that  is  used  to  develqp  the 
softwcupe  system. 

3.  Man  interacticxi  [R,  DefaiiLt  =  Average,  Required]  -  level  of 
man  interaction  inherent  in  the  system. 

4.  Timing  and  Criticality  [R,  Default  =  Average,  Required] - 
performance  requirements  related  to  response  times. 

5.  Software  testability  [R,  Default  =  Average,  Required]  -  level 
of  difficulty  associate  with  extent  and  ease  of  software 
testing. 

6.  Software  interfaces  [R,  Default  =  Average,  Required]  -  number 
of  external  software  interfaces  to  other  programs,  systems, 
and/or  peripheral  camanications  equipment  required  by  the 
software  system. 

7.  Embedded  development  system  [R,  No  default,  Required] - 
extent  to  which  hardware  will  be  developed  concurrently  with 
the  software. 

8.  Pcogranming  language  ocnplexity  [R,  No  default,  Required] - 
oorplexity  of  the  principle  software  programming  language 
rated  according  to  the  difficulty  of  language  constructs, 
syntax,  and  required  training. 

Sizing  Factors 

Note:  SASET  will  estimate  size  according  to  the  vjser  defined 

functioned  ity  of  a  software  system  based  upon  the  associated 

databeise.  The  user  may  cptionedly  bypass  the  sizing  by  software 

functionedity  eind  directly  irput  sizing  inforination. 

Items  1  throu^  5  refer  to  the  "Software  Functionality"  inputs. 

1.  Software  Function  [C,  No  default,  Required  for  functional 
sizing]  -  lowest  level  of  decotposition  for  the  cotponent  of 
the  software  system  to  be  developed.  The  user  enters 
quantitative  parameters  (items  2  thrai^  5)  for  each  software 
function  defined. 
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2.  Size  value  [R,  No  default,  Required  for  functional  sizing] - 
perceived  conplexity  and  size  relative  to  the  base  size 
estimate  provided  for  the  identified  software  function. 

3.  Generated/operational  function  [C,  No  defaxilt,  Required  for 
functional  sizing]  -  indicates  whether  the  software  function 
is  provided  by  seme  element  of  the  operationcil  environment  or 
if  the  function  needs  to  be  designed  cind  developed  from 
scratch  by  the  software  developer. 

4.  Oode  condition  [%,  No  default.  Required  for  functional 
sizing]  -  the  portion  of  software  function  that  is  a  new 
function,  modified,  or  a  rehosted  function  requiring  a 
smaller  portion  of  the  life  cycle  for  inplementation  than  a 
modified  function. 

5.  language  [%,  No  default.  Required  for  functional  sizing] - 
percent  of  the  softweire  func±ion  vAiich  is  to  be  iirplesnented 
in  hi^-order  language  and  the  percent  to  be  iitplemented  in 
assembly  language. 

6.  CEO  Host  [#,  No  default.  Required  for  functioncil  sizing]  -CEO 
number  of  the  total  number  of  system  CEOs  on  vtiich  the 
software  function  is  to  be  hosted. 

Note:  Items  7  through  13  refer  to  direct  sizing  inputs. 

7.  Software  type  [C,  No  default.  Required  for  direct  sizing] - 

one  of  four  software  types  (system,  application,  support,  and 
security)  for  which  the  condition  of  the  code  (new,  modified, 
rehosted  items  8  throui^  13)  must  be  identified. 

8.  New  HaL  oode  [#,  No  default.  Inquired  for  direct  sizing] - 

ameunt  of  HOL  software  that  is  to  be  developed  from  scratch. 

9.  Modified  HOL  oode  [#,  No  default.  Required  for  direct  sizing] 

-  amount  of  HOL  software  which  has  some  development  already 
complete  and  requires  retesting,  seme  redesign  and  recoding 
effort  in  order  to  be  utilized. 

10.  Rshosted  HOL  oode  [#,  No  default,  I^equired  for  direct  sizing] 

-  amount  of  conpleted  and  tested  HOL  software  code  vhich  is 

to  be  transferred  from  one  cenputer  system  to  another. 

11.  New  Assembly  oode  [#,  No  default.  Inquired  for  direct  sizing] 

-  amount  of  assembly  code  that  is  to  be  developed  from 
scratch. 
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12.  Modified  Assembly  cxade  [#,  No  defaxilt,  Required  for  direct 
sizing]  -  amount  of  assembly  code  vtiich  has  some  develc^xnent 
already  complete  and  which  can  be  utilized  in  the  software 
program  under  consideration. 

13.  Rehosted  Assenobly  code  [#,  No  default,  Required  for  direct 
sizing]  -  amount  of  cissembly  software  viiich  has  some 
development  already  ccrplete  and  requires  retesting,  some 
redesign  and  recoding  effort  in  order  to  be  utilized. 

E’rocess  Attributes 

1.  Schedule  [S,  No  default.  Optioned]  -  Start  date,  finish  date, 
and  date  of  Critical  Design  Review  relative  to  the  single 
CSd  being  estimated.  SASET  generates  a  schedule  based  an 
user-srpplied  data  in  additicMi  to  an  ''optimal”  schedule  based 
on  the  start  date  that  is  input. 

2.  System  requirements  [R,  Default  =  Average,  Required]  -  extent 
to  v*iich  system  requirements  are  well  defined  and  understood. 

3.  Software  requirements  [R,  Default  =  Average,  Required] - 
extent  to  viiich  software  requirements  are  well  defined  and 
understood. 

4.  Software  documentaticn  [R,  Default  =  Average,  Required] - 
level  and  volume  of  software  documentation  required  for  the 
project. 

5.  Software  develcpment  tools  [R,  No  default,  Required]  -  level 
of  software  development  tools  availability  ranging  from  basic 
debuggers  eind  full  screen  editors  to  scphisticated  program 
code  generators  that  greatly  enh£a>ce  productivity. 

6.  ’Technology  impacts  [R,  Default  =  Average,  Required]  -  extent 
to  vtiich  advances  in  technology  will  lead  to  a  mo^fication 
of  system  or  software  requirements  during  the  development. 

7.  cars  software  [R,  Default  =  Average,  Required]  -  level  of 
effort  required  to  integrate  off-the-shelf  software  with  the 
operational  software  that  is  being  develcp^ed. 


CamDuter  Attributes 

1.  Hardware  system  type  [R,  No  default,  Required]  -  hardware 
configuration  of  the  system  on  which  the  softwcure  will  be 
hosted. 
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2.  Cbre  utilized  [%,  No  default,  Required]  -  amount  of  machine 
internal  storage  that  is  utilized  by  the  operational 
softwcire. 

3.  Hbrkstaticn  types  [#,  No  default.  Required]  -  number  of 
workstation  types  requiring  special  screen  clearing  and  set¬ 
up  operations. 

4.  MicrcpfEOoessQr  cxxle  [%,  Default  =  Average,  Required] - 
percentage  of  the  software  functions  (with  respect  to  the 
total  softwcire  job)  that  cure  to  be  firmware. 

5.  Hardware  ocnstzaints  [R,  Default  =  Average,  Required]  - 
degree  to  which  the  software  system  utilizes  available 
processor  memory,  speed,  and  througiput. 

6.  Etevelopnent  versus  host  system  [R,  Default  =  Average, 

Required]  -significance  of  differences  between  the 
development  hcurdweure  system  and  the  host  system. 

Personnel  Attributes 

1.  Hardware  experience  [R,  Default  =  Average,  Required] - 

conpeiny's  experience  with  the  hosting  develcpment  hardware 

equipment. 

2.  Software  experience  [R,  Default  =  Average,  Required] - 

company's  ej^^erience  with  the  software  langiaage  and  operating 
system. 

3.  Oevelq^™^^  team  [R,  Default  =  Average,  Required] - 

develcpment  team's  level  of  familiarity  with  the  type  of 
function  to  be  develcjped. 

Environmental  Factors 

1.  Develcpment  Icxzitions  [#,  No  defaiolt.  Required]  -  the  number 
of  locations  at  which  the  software  is  to  be  develcped. 

2.  Customer  locatlcns  [#,  No  default,  Recjuired]  -  number  of 

(Customer  Icxations  recjuiring  the  transfer  of  software 

develcpment  information  or  reviews  during  the  software 

develcpment  cycle. 

3.  Security  level  [R,  No  default.  Required]  -  level  of  computer 
security  rec^uirements  that  control  acxess  to  system  cxxie, 
ciata,  cind  software  functions. 

4.  InfcoBtiai  travel  requirements  [R,  Default  =  Average, 

Required]  -  amount  of  information  travel  recjuired  between 
custcjmer  and  contractor  locations. 
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5.  Perscnnel  rescxiroes  [R,  No  defaxiLt,  Required]  -  relative 
difficulty  for  staffing  of  the  project  due  to  special 
training  and  security  requirements  of  project  personnel. 

6.  Developnent  facilities  [R,  Default  =  average.  Required] - 
address  the  characteristics  of  the  developnnent  facilities  and 
the  perceived  availability  of  computer  hardware. 
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Prociuct  Attributes 

1.  IVP®  of  software  [C,  Default  =  other.  Required]  -  application 
specific  donain  of  software  to  be  developed. 

2.  Ada  usage  fetctor  [R,  DefaiiLt  =  ncmincil,  Required] - 
percentage  of  the  software  being  developed  that  will  be 
written  in  the  Ada  prograinming  language. 

3.  Product  ocB^exity  [R,  Default  =  nominal,  Required]  -  level 
of  ocrplexity  of  the  module  to  be  develop^  as  a  fuixtion  of 
the  processing  logic,  optimization  and  efficiency  concerns. 

4.  Degree  of  real-time  [R,  DefaiHt  =  nomincil.  Required]  -  the 
amount  of  tasking  required  to  meet  the  system's  performance 
requirements. 

5.  Degree  of  reuse  [R,  Default  =  nominal.  Required]  -  percentage 

of  software  that  will  be  packaged  for  reuse  on  other 

applications. 

Sizing  Factors 

1.  Database  size  [R,  Default  =  ncmdr.al,  Required]  -  relative 

database  size  (in  bytes) ,  represented  as  a  percentage  of  the 
total  program  size. 

2.  Subprojects  [#,  No  default.  Required]  -  number  of 

sul^rojects/deliverables/CSCI's  that  are  defined  to  be  part 
of  the  project.  Sut^rojects  are  defined  if  a  s^^arate 

estimate  is  to  be  developed  for  each  of  them. 

Note;  The  project  may  be  sized  using  either  a  function  point 
count  or  source  lines  of  code  count  (SLDC) . 

Items  3  through  7  eu^e  required  for  function  point  sizing 

3.  Function  Paint  Oount  [#,  No  default.  Required]  -  raw  furjction 
point  count  cedculated  for  the  software  system  based  on  the 
number  of  external  inputs,  outputs,  inquiries,  interfaces, 
internal  files,  operating  modes,  rendezvous,  and 
stimulus/ response  relationships. 

4.  OperancV^^P^i^tar  Count  [#,  No  default.  Required]  -  measure  of 
the  size  consumed  by  the  mathematical  equations  specified  for 
the  systen. 

5.  language  type  [C,  Default  =  Ada,  Required]  programming 
Icuiguage  which  will  be  used  to  develop  the  software. 
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6.  Architec±ural  factor  [E,  Default  =  1.00,  Required]- 

adjustment  vSiich  accounts  for  the  inpact  of  alternative 
system  ctrchitectures.  Ihe  value  is  calculated  by  the  ASSEJT-R 
package. 

7.  Technology  factor  [E,  Default  =  1.00,  Required]  -  adjustment 
vhich  accounts  for  the  experience  base  and  capabilities  of 
the  software  develcpment  organization.  Hie  value  is 
calculated  by  the  ASSET-R  package. 

Note:  Items  8  throu^  13  are  required  for  SIDC  sizing 

8.  Ada  oaipoiients  size  (new)  [#,  No  default.  Required]  -  lines 
of  new  Ada  code  to  be  develcped  by  this  project  including 
instantiations  and  program  specification. 

9.  Ada  ocnpcnents  size  (reused)  [#,  No  default.  Required] - 
lines  of  Ada  code  reused  as-is,  without  modification.  Each 
instantiated  coiponent  should  te  specified  once,  regcudless 
on  the  number  of  times  it  will  be  invoked. 

10.  Ada  components  size  (modified)  [#,  No  default,  Recjuired]- 
lines  of  existing  Ada  cxde  reused  as  ein  Ada  component,  but 
modified  during  incorporation  into  the  software  system. 

11.  Other  ocnpcnents  size  (rmf)  [#,  No  default,  Required]  -  lines 
of  new  cxxle,  written  in  a  language  other  than  Ada,  that  will 
be  incorporated  as  part  of  the  software  systen. 

12.  Other  cxmpcnents  size  (reused)  [*,  No  default,  Recjuired]- 
lines  of  cede  written  in  a  language  other  than  Ada,  reused  as 
a  conponent  and  incsorporated  into  the  software  system, 
withexit  modification. 

13.  Other  cxnpcmients  size  (modified)  [#,  No  default.  Required] - 
lines  of  code  written  in  a  language  other  than  Ada,  reused  cis 
a  exmponent  but  modified  during  incorporation  into  the 
software  system. 

Process  Attributes 

1.  Required  develcpnent  schedule  [R,  Default  =  ncmincil. 
Required]  -  level  of  schedule  (constraint  imposed  on  the 
project  team  developing  a  software  subsystem  expressed  in 
terms  of  the  percentage  of  sciiedule  stretchout  or 
acceleration  with  respeert  to  a  ncminal  schedule  with  ncminal 
effort. 

2.  Degree  of  standard! zation  [R,  Default  =  nomiral,  Reejuired]- 
degree  to  vhich  lantguage,  life  c^cle,  and  military  standards 
are  applied  to  the  software  development  effort. 
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3.  Scope  of  sipport  [R,  Defavdt  =  ncminal,  Required]  -  level  of 
sv;?:port  that  the  developer  will  provide  to  non-software 
organizations. 

4.  Use  of  modem  software  methods  [R,  Defaxilt  =  nominal, 
Required]  -  type  of  softwaure  methods,  such  as  structured 
programming  or  object-oriented  design,  that  will  be  used 
during  the  software  development  effort. 

5.  Use  of  peer  revieuB  [R,  Default  =  nominal,  Required]  -  the 
types  of  reviews  that  will  be  used  during  the  software 

,  development  effort. 

6.  Use  of  software  tools/enviroimients  [R,  Default  =  ncminal, 
Reqiiired]  -  level  of  software  tools  and  tool  environments 
used  in  the  development  process  ranging  from  beisic  Ada 
language  tools  to  a  full  integrated  life  cycle  APSE. 

7.  Requirements  volatility  [R,  Default  =  ncminal.  Required] - 
percentage  of  requirements  that  are  well  established  and  will 
not  change. 


Computer  Attributes 

1.  System  architecture  [C,  Default  =  centralized  architecture 
using  a  single  processor.  Required]  -  hardware  configuration 
of  the  system  on  which  the  software  will  be  hosted. 

2.  Software  tools/envirormient  stability  [R,  Default  =  nomincd, 
Required]  -  stability  end  capability  of  the  coirpiler  end  the 
frequency  of  changes  to  the  softwcure  tools  and  tools 
environments  used  in  development. 

3.  Degree  of  optimization  [R,  Default  =  ncminal,  Required] - 
degree  to  vhich  available  processor  memory,  speed,  and 
irput/out^xrt  eire  utilized. 

Personnel  Attributes 

1.  Ada  prcjjects  ocnpleted  [#,  No  defavilt.  Required]  -  combined 
average  of  the  number  of  Ada  projects  ccnpleted  by  members  of 
the  project  team. 

2.  AnELlyst  capability  [R,  Default  =  ncminal.  Required]  -  level 
of  capability  of  the  analysts  expressed  in  terms  of 
percentiles  with  respect  to  the  overall  population  of 
analysts. 


&-16 


3.  Applicaticns  experienae  [R,  Default  =  ncminal,  Required] - 
average  experience  the  cUT2d.ysts  have  with  the  types  of 
applications  within  the  software  system. 

4.  Ada  enviroment  experience  [R,  Default  =  ncminal,  Required]- 
average  e^^jerience  the  analysts  have  with  the  Ada  software 
develcproent  environment  that  will  be  used  during  develc^xnent 
of  the  softwaure. 

5.  Ada  language  experience  [R,  Default  =  nominal,  Required] - 
average  experience  the  anaili^ts  have  with  the  Ada  programming 
language. 

6.  Ada  methodology  esqxrienoe  [R,  Default  =  ncminal,  Required]- 
average  experience  the  analysts  have  with  the  chosen  Ada 
development  methodology. 

7.  Team  capability  [R,  Default  =  ncminal.  Required]  -  type  of 
team  -•  design,  prograitming,  or  interdisciplinary,  to  be  used 
for  the  software  develcpment  effort. 

Environmental  Factors 

1.  Software  ongfanizaticns  involved  [#,  No  default.  Required] - 

organizations  which  provide  level  of  effort  support  (i.e. 
configuration  management  and  quality  assurance)  and  system 
level  support. 

2.  Organizational  internee  ocnpl^dty  [R,  Default  =  ncminal. 

Required]  -  number  of  internal  and  external  interfaces 
between  all  involved  personnel  and  the  degree  to  vitiich 
organizations  involved  in  the  effort  are  geographically 
dispersed. 

3.  Security  requirements  [R,  Default  =  ncmiral.  Required] - 

security  measures  that  must  surround  the  develcpment  effort. 

4.  Resource  availability  [R,  Default  =  ncminal.  Required] - 

percentage  of  staff  availability  assigned  to  the  project  and 
accessibility  to  software  tools  and  equipment. 
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Product  Attributes 

1.  Estimate  type  [C,  No  default,  Pequired]  -  indicates  whether 
the  project  being  estimated  is  a  new  program,  an  enhancement, 
or  a  maintenance  change. 

2.  Estimate  scope  [C,  No  default.  Required]  -  the  ]dnd  of 
program  or  system  that  the  estimate  is  for:  prototype,  module 
of  a  prxsgram,  reusable  module  for  multiple  programs,  corplete 
stand-alone  program,  system,  release,  etc. 

3.  Project  class  [C,  No  default.  Required]  -  indicates  whether 
the  software  is  to  be  used  intemcilly  or  externally,  at 
single  or  multiple  locations,  developed  under  commerciad, 
government,  or  military  cxxjtract,  and  used  by  others  in  the 
public  dcmain  or  sold  as  a  retail  product. 

4.  Project  type  [C,  No  default.  Required]  -  application  domain 
of  the  software  to  be  developed. 

5.  New  code  language  [C,  No  default.  Required]  -  source  language 
in  vhich  new  software  is  to  be  develc^jed. 

6.  New  oode  language  level  [R,  Default,  Required]  -  the  number 
of  assembly  language  statements  it  will  take  to  create  the 
same  function  that  one  statement  will  take  in  the  new  oode 
language. 

7.  New  oode  logical  ocnplexity  [R,  Default  =  Average,  Required] 
-  rating  from  1  (mostly  siuple  algorithms)  to  5  (many  conplex 
calculations)  for  level  of  complexity  of  the  problem  or 
algorithms  to  be  coded. 

8.  New  oode  structural  oaplexity  [R,  Default  =  W^l  Structured, 
Required]  -rating  from  1  (norprocedural)  to  5  (poor  structure 
with  many  conplex  paths  and  modules)  for  new  program 
structure. 

9.  New  code  data  ocxplexitY  [R,  Default  =  Multiple  Files, 
Fields,  and  Data  Interactions,  Required]  -  rating  from  1 
(sinple  data  with  few  variables)  to  5  (very  conplex  data 
elements  and  data  interaction)  for  ccnplexity  of  both  the 
file  structure  and  the  data  elements  viiich  the  program  or 
system  will  utilize. 

10.  Base  code  language  [C,  No  default,  Required  for  erhancement 
or  maintenance  estimate  type]  -  refers  to  existing  code  of  a 
program  or  system. 
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11.  Base  cxxle  language  level  [R,  Default,  Required  for 
enhanoeznent  or  maintenance  estdjnote  type]  -  the  nund^er  of 
assanbly  language  statesnents  it  will  take  to  create  the  same 
functic^  that  one  stateinent  will  take  in  the  base  cx3de 
language. 

12.  Base  logical  ocnplexity  [R,  Default  =  Average,  Required  for 
enhancement  or  maintenance  estinate  type]  -  rating  from  l 
(mostly  siitple  algorithms)  to  5  (many  ccnplex  calculations) 
for  level  of  ccnplexity  of  the  pi^lem  or  eilgorithms  of  the 
bctse  system. 

13.  Base  oode  structural  ocnplexity  [R,  Default  =  Well 
Structured,  Required  for  enhancement  or  maintenance  estimate 
type]  -  rating  from  1  (norprocedural)  to  5  (poor  stnacture 
with  many  ccnplex  paths  and  modules)  for  base  program 
structure. 

14.  Base  code  data  ocnplexity  [R,  Default  =  Multiple  Files, 
Switches,  and  Data  Interactions,  Required  for 
enhancement  or  maintenaix»  estimate  type]  -  rating  from  1 
(sinple  data  with  few  variables)  to  5  (very  ccnplex  data 
dements  and  data  interaction)  for  ccnplexity  of  both  the 
file  structure  eind  the  data  elonents  which  the  base  program 
or  system  utilizes. 

15.  Reusable  code  language  [C,  No  default.  Required  if  souroe 
code  reusability  >  0%]  -  refers  to  code  that  will  be  reused 
from  an  existing  application. 

16.  Reusable  oode  language  level  [R,  default.  Required  if  souroe 
code  reusability  >  0%]  -  the  number  of  assembly  language 
statements  it  will  tzike  to  create  the  same  function  that  one 
statement  will  take  in  the  reusable  code  language. 

Sizing  Factors 

1.  Source  oode  reusability  [R,  Default  =  Moderate  (>  25%), 
Required]  -  rating  from  1  (extensive,  >  75%  use  of  reusable 
code)  to  5  (no  use  of  reusable  code)  for  the  amount  of 
included  oode  frcm  software  libraries. 

2.  Reusable  oode  size  [#,  No  default.  Required  if  souroe  oode 
reusability  >  0%]  -  refers  to  reused  source  oode. 

3.  Base  oode  size  [#,  No  default.  Required  for  enhancement  or 
maintenance  estimate  type]  -  refers  to  existing  code  of  a 
program  or  system. 

NOTE:  SPQFV20  will  optionally  estimate  new  oode  size  using  the 

function  point  methodology  if  a  SWC  value  is  not  entered 
directly  for  new  oode  size. 
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4. 


New  ac3de  size  [#,  No  default.  Required  for  direct  sizing] - 
refers  to  new  code  to  be  developed.  The  line  of  code  count 
includes  procedural  statesments  and  data  definitions  and 
excludes  ccxninents,  JCL,  and  included  code. 

NCfTE:  Items  5  throm^  9  are  required  inputs  for  function  point 

sizing 

5.  Inputs  [#,  No  defaiiLt,  Required  for  function  point  sizing] - 
unique  number  of  input  file,  control  information,  or  input 
screens  that  enter  a  program  from  an  extemed.  source.  Irputs 
are  counted  if  they  require  unique  processing  logic  or 
introduce  new  formats. 

6.  Outputs  [#,  No  default.  Required  for  function  point  sizing] - 
unique  number  of  output  files,  control  information,  or  output 
screens  that  leave  a  program  and  go  to  an  external  source. 
Outputs  aue  counted  if  they  require  unique  processing  logic 
or  introduce  new  formats. 

7.  Inquiries  [#,  No  default.  Required  for  function  point  sizing] 
-  unique  iiput/oulput  ccmbination  sucii  as  a  HELP  screen, 
selection  menu,  or  inquiry  \diere  an  input  is  entered  to 
direct  a  search  of  internal  files  and  generate  an  immediate 
output.  Inquires  are  counted  if  they  require  unique 
processing  logic  and  cause  no  change  to  internal  data. 

8.  Data  files  [#,  No  default.  Required  for  function  point 
sizing]  -  number  of  irput  and  output  files  v^iich  the  program 
will  generate,  access,  or  update.  Also,  each  hierarchiccil 
path  through  a  database  or  table  in  a  relational  database 
that  requires  unique  processing. 

9.  Interfaces  [#,  No  default.  Required  for  function  point 
sizing]  -  files  or  datcibases  passed  between  or  shared  among 
separate  applications. 

Process  Attributes 

1.  Project  estimating  goals  [C,  No  default.  Required]  -  user 
iirposed  constraints  to  be  factored  into  the  SPQFV20  estimate 
relating  to  staff  size,  schedule,  cost,  and  required  quality 
and  reliability. 

2 .  Requixenents  def ixiiticn  [R,  Default  =  Fairly  Clear  User 
Requirements,  Required]  -  rating  frcm  l  (program  developers 
are  clLso  users  of  the  program)  to  5  (uncertain,  changing, 
ambiguous  requirements)  indicating  the  level  of  clarity  and 
cotpleteness  of  requirements  definition. 
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3.  Program  design  autcnaticn  [C,  Default  =  New  Designs  with 

Partied  Text/Graphics  Support,  Required]  -  rating  frem  1 

(reusable  designs  with  automated  text/graphics  si:?:port)  to  5 
(new  designs  or  hasty  design  with  no  autcciation)  signifying 
the  level  of  program  design  automation. 

4.  User  documentation  [C,  Default  =  Programmers  or  Users; 
Automated  Gre^jhics/Text  Support,  Reqwiired]  -  identifies 
whether  professiored  writers,  programmers  or  users  write 
documentation  and  the  amount  of  documentation  automation 
utilized. 

ConiDuter  Attributes 

1.  *Derminal  re^xree  time  [C,  Default  =  1-5  Seconds,  Required]- 

average  response  time  in  seconds  from  the  time  the  Return 

(Enter)  key  is  depressed  until  the  termined  responds. 

Personnel  Attributes 

Note;  Maximum  and  minimum  staff  size  cure  selected  by  the  user  if 
project  estimating  goals  stipulate  constrained  staffing. 

1.  (bodimmi  staff  size  [#,  No  default,  c^ional]  -  user  iitposed 

staff  size  constraint  to  be  factored  into  the  SPQIV20 

estimate. 

2.  Minimum  staff  size  [#,  No  default,  c^tional]  -  user  imposed 

staff  size  constraint  to  be  factored  into  the  SPQIV20 

estimate, 

3.  Stzdf  availability  [%,  Default  =  100%,  Required]  -  average 
time  of  availability  of  project  personnel  to  be  assigned  to 
the  project. 

4.  Average  work  week  [#,  Default  =  40  hours.  Required]  -  average 
work  week  for  the  staff  on  the  project  being  estimated. 

5.  Average  work  year  [#,  Default  =  220  days.  Required]  -  average 
number  of  days  in  a  work  year  for  the  organization. 

6.  Stadf  experience  [R,  Default  =  Even  Mixture  of  Experts  and 
New  Hires  or  Novices,  Required]  -  ratio  of  experienced 
project  personnel  familiau:  with  the  program  to  new  hires  and 
novices  new  the  application. 
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Environmental  Factors 


1.  Project  novelty  [C,  Defaxilt  =  Even  Mixture  of  New  and 
R^jeated  Features,  Required]  -  novelty  of  application  to  the 
organization. 

2.  Office  facilities  [C,  Default  =  Multi-Eliployee  Offices, 
Required]  -  physiceil  environment  where  software  development 
takes  place  such  as  individual  offices  with  excellent 
facilities  or  an  open  office  arrangement. 
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Product  Attributes 

1.  A{plicaticn  aaif>leadty  [R,  Default  =  nominal,  Required] - 

inherent  difficulty  associated  with  the  application  under 
estimation  considering  the  number  of  years  of  work  or  study 
required  to  become  proficient  in  the  application  area. 

2.  Di^ilay  rec|airements  [R,  DefaiiLt  =  ncmincil,  Required]- 

amount  of  extra  effort  required  to  interface  with  the  user. 

3.  Language  type  [R,  Default  =  nominal,  Required]  -  inherent 

complexity,  magnituie,  and  leeuning  difficulty  of  the 

development  prograinning  language. 

4.  Real  time  operation  [R,  Default  =  nominal.  Required]-* 

percentage  of  the  develcpnent  task  that  must  perform  its 
processing  function  within  absolute  time  constraints. 

5.  Time  constraint  [R,  Default  =  nominal.  Required]  -  percentage 
of  code  that  must  have  special  attention  to  ensure  adequate 
time  performance  (not  real  time)  in  order  to  meet  overall 
system  performance  requirements. 

Sizing  Factors 

1.  Source  lines  of  code  [#,  No  default,  Required]  -  amount  of 
programmer  developed  source  lines  of  code  including  all 
executable  souroe  lines,  data  declarations,  and  format 
statements  and  excluding  comments  and  continuations. 

Process  Attributes 

1.  Modem  developmoit  practices  [R,  Default  =  nominal,  Required] 
-  degree  to  which  modem  development  practices  are  used  in 
developing  the  software. 

2.  Quality  assurance  level  [R,  Default  =  nominal,  Required]- 
required  quality  assurance  effort  based  on  the  inpact  of  a 
software  failure. 

3.  Requirements  volatility  [R,  Default  =  nominal.  Required] - 
frequency  and  criticality  of  anticipated  changes  to  the 
software  requirements  specification  during  development. 

4.  ^3ecificaticn  level  [R,  Default  =  nominal.  Required]  -  level 
of  documentation  that  will  be  produced  during  the  project 
development  usually  consistent  with  quality  assurance  level 
and  test  level. 


B-23 


5. 


Test  level  [R,  Defaiilt  =  nceinal,  Required]  -  d^>th  to  vAuch 
the  software  will  undergo  testing  through  Final  Qualification 
Test  usually  based  oti  the  iirpact  of  a  software  failure  during 
operation. 

6.  Tool  sappcact,  autcmated  [R,  Default  =  naninal,  Required]- 
availability  and  use  of  autcroated  tools  by  the  development 
team  ranging  from  primitive  to  the  advanced  APSE. 

ConiDuter  Attributes 

1.  Mebcc^  constrzdnts  [R,  Default  =  nominal.  Required]  -  effort 
recjiired  to  make  the  software  function  within  available 
ccitputer  memory. 

2.  Rdiosting  effort  [R,  Default  =  ncmined.,  Required]  -  effort 
required  to  convert  the  software  from  the  development 
facilities  to  the  target  system  based  on  the  extent  of 
language  and  system  changes  required. 

3.  Resource  dedication  [R,  Default  =  nominal.  Required] - 
percentage  of  availability  of  the  virtual  machine  to  the 
software  development  project. 

4.  Virtual  machine  complexity  [R,  Default  =  nominal.  Required] - 
relative  difficulty  to  fully  understand  all  virtual  machine 
features. 

5.  Turnaround  -  Logon  throu^  heu:doopy  [R,  Default  =  nomin2d. 
Required]  -  average  time  required  to  log  onto  the  system, 
perform  ccmmcind  actions  to  edit  and  save  a  file,  compile, 
link,  execute,  print  a  source  listing,  and  receive  a 
hardcopy. 

6.  Virtual  machine  volatility  [R,  Default  =  nominal.  Required] 
-  frequency  and  criticality  (major/minor)  of  changes  to  the 
virtual  machine  during  develcproent. 

Personnel  Attributes 

1.  Analyst  capability  [R,  Default  =  ncmincLL,  Required]  -  level 
of  capability  of  the  analj^ts  rated  according  to  team 
efficiency,  motivation,  attitude,  quality  of  previous  work, 
ability  to  ccmmunicate,  and  ability  to  learn  quickly. 

2.  Analyst  appUcaticns  experience  [R,  Default  =  nominal. 
Required]  -  the  project  team's  average  level  of  experience 
with  similau:  applications. 

3.  language  experience  [R,  Default  =  nominal,  Required]  -  the 
project  team's  average  level  of  experience  with  the 
progranming  language  to  be  used. 
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4.  Progranner  c^ability  [R,  Default  =  noninal,  Required] - 
level  of  capability  of  the  programrners  based  on  ability  to 
define  design  details,  develop  code,  prepare  test  cases  for 
program  modules,  and  integrate  modules. 

5.  Virtual  machine  e9q)erienoe  [R,  Default  =  ncminal,  Required]- 
the  project  team's  average  level  of  experience  with  the 
underlying  virtual  machine  for  an  application. 

Environmental  Factors 

,  1.  Multiple  site  develcpnent  [R,  Default  =  ncriineQ.,  Required]- 
additional  time  required  for  development  due  to  the  nurttoer 
and  location  of  different  development  orgcinizations. 

2.  Resouroe  location  [R,  E^efaiilt  =  ncminal,  Required]  -  degree 
of  access  to  develcproent  resources,  system  consultants, 
programming  language  support,  and  software  tools  support. 
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HARDNASE  RBQUIREMENrrS 


Table  C-1  suitmarizes  the  hardware  reqLiirements  for  each  of  the 
models.  All  of  the  models  are  available  on  an  lEM  PC  (or  coipatible) . 
Additional  details  concerning  hardware  requirements  are  provided  in  the 
follcwing  text. 


TABLE  C-1 

HARDWARE  REQUIREMENTS 


lEM  PC 

ZENITH-248 

PRIME 

VAX 

MODEM 

OOSDODL 

X 

PRICE  S 

X 

X 

X 

SASET 

X 

SoftCost-Ada 

X 

X 

SPQIV20 

X 

SYSTEM-3 

X 

X 

L  -  -  -  -  ■  -  - 

_ 1 

X  =  Available  to  DoD  and  Commercial  users 
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O06IICX3L:  COSTMODL  runs  on  IBM  PCS  and  ccnpatibles.  A  hard 
disk  and  640K  bytes  of  memory  are  required.  Any  monitor  may 
be  used,  but  a  color  monitor  is  preferred  since  color  is  used 
to  differentiate  between  different  classes  of  data. 

2.  IKCCE  S:  FRTCE  S  runs  on  a  reiME  miniccnpiter  cperating 
under  FRIMDS.  In  addition,  PRICE  S  can  be  accessed  via  a 
time-sharing  system  with  an  office  terminal  and  standard 
modem. 

3.  SASET:  SASET  may  be  hosted  on  any  IBM  PC  or  ccnpatible  with 
a  minimum  of  512K  bytes  of  memory,  one  disk  drive,  and  an 
8088/86,  80186,  80286,  80386  microprocessor  running  K-DOS  or 
MS-rX)S,  version  2.0  or  hi<3^er.  Ihe  model  functions  with 
either  a  color  or  monochrcroe  monitor.  A  hard  disk  eind 
printer  are  optional. 

4.  SoftOost-Ada:  SoftCost-Ada  runs  on  an  lEM  PC,  PC/XT,  PC/AT, 
EB/2  or  conpatible  with  a  minimum  of  256K  bytes  of  memory  and 
a  color  or  monochrome  display.  The  system  requires  PC-DOS  or 
MS-DOS,  version  2.0  or  hi^er.  A  minimum  of  one  floppy  disk 
drive  is  required.  A  hard  disk  drive  and  printer  are 
optional.  SoftCost-Ada  may  also  be  hosted  on  the  Digital 
MicroVax  II  or  VAX  11/780  with  VMS  version  4.6  or  hic^er. 

5.  SPC5V20:  SPQR/20  runs  on  an  IBM  PC,  XT,  AT,  or  conpatible 
with  512K  bytes  of  memory  and  a  color  or  monochrcme  display. 
Two  floppy  disk  drives  or  a  floppy  disk  oirive  and  a  hard  disk 
oirive  are  recjuired. 

6.  SYSEEM-3:  SYSTEM-3  runs  on  an  lai  PC,  XT,  AT,  Zenith  248  or 
oxmpatibles  with  a  minimum  of  512K  bytes  of  memory  and  a 
color  or  monoxhrrjne  oiisplay.  The  system  recjuires  PC-DOS  or 
MS-DOS,  version  2.0  or  hi^ier.  A  minimum  or  one  floppy  disk 
drive  is  required. 
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GCNlBACimL  ARRANCaiEmS  AMD  COBJS 

Tables  D-1  and  D-2  summarize  the  availability  and  cxjst  of  models 
applied  in  the  test  case  study.  Table  D-1  summarizes  the  availability 
of  each  model  to  DoD  and  ccnimercial  users.  Availability  throu^ 
request  means  that  a  potentieil  user  can  receive  the  model  at  no  cost  by 
contacting  the  model  POC.  The  model  is  received  on  disloettes  that  are 
provided  by  the  requesting  agency.  Table  D-2  shows  the  DoD  rates  for 
each  of  the  models.  S^jarate  ccnimercicil  rates  apply  to  PRICE  S  and 
Systeni-3.  Additional  costs  may  be  associated  with  user  training,  which 
is  required  for  seme  of  the  models.  Also,  rates  will  vary  d^aending 
upon  the  type  of  licensing  agreement  procured  (annual,  site,  corporate, 
etc.) . 


1.  OOSOfXlL:  OOSTMDDL  is  available  to  eLLl  DoD  and  ccratiercied. 
users.  Version  5.0  of  ODSDODL  is  available  by  ocxitacting 
Bemie  Itoush  at  NASA-  Johnson  Space  Center.  Versic«i  5.0 
inplements  the  complete  Ada  GOOCMD  model  and  the  Incremental 
Develcproent  model  which  were  intoduoed  by  Dr.  Boehm  at  the 
November  1987  OOOCMO  User's  Group  Conference.  It  does  not 
include  the  enhancements  to  the  Ada  model  that  Dr.  Boehm 
introduced  at  the  1988  OOOOMD  User's  Group  Conference. 
Enhancements  will  be  incorporated  in  Version  6.0.  Requests 
should  be  accempanied  with  three  360K  5.25"  disks. 
Inplementors  are  currently  soliciting  feedback  on  the 
package's  users-interface.  After  upgrades,  OOSTMDDL  will  be 
available  from 

NASA/COSMIC 

The  University  of  Georgia 
Ccsiputer  Services  Annex 
Athens,  GA  30602 
(404)  542-3265 

There  is  a  nominal  handling  charge  for  the  program. 

2.  PRICE  S:  PRICE  S  is  part  of  the  PRICE  system  of  models  that 
includes  PRICE  SZ  for  software  sizing,  and  PRICE  SL  for 
software  life  cycle  costs  (maintenance,  enhancement,  and 
growth  activities  associated  with  life-cycle  support) . 
Government  users  can  use  the  PRICE  S  package  cxi  a  time 
sharing  basis  at  $82  per  hour  (throu^  11/91)  by  contacting 
Lt.  Ken  Nelson  of  the  Aeronautical  Systems  Division  at  Wri^t 
Patterson  AFB.  Cemmercial  users  can  use  the  PRICE  S  package 
on  a  time  sharing  basis  at  $15/hour  of  connect  time  and 
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$0.060/resource  vmit  of  CPU  time  by  contacting  PRICE  Systems 
at  Moorestown,  New  Jersey,  but,  they  must  also  pay  an  access 
fee  of  $40, 000/year  for  one  unit  or  $60, 000/year  for 
unlimited  access.  Ccninercial  users  can  also  lease  the  PRICE 
S  package  for  installatiofi  on  their  cwn  PRIME  miniconputer  at 
$60, 000/year  for  one  user  at  a  time  or  $80, 000/year  for 
unliriited  access.  A  one  week  training  course  is  mandatory 
and  costs  $1,312.50  (thrcu^  11/91)  for  a  government  student 
or  $1,750  for  a  ccnroercial  student.  These  costs  include 
refresher  training,  manucil  updates,  technical  assistance,  and 
newsletter  at  no  additioned  charge. 

3.  SASET:  SASET  is  presently  being  used  by  the  U.S.  Air  Force 
Cost  Center  and  the  Naval  Center  for  Cost  Aralysis. 
Availability  to  DoD  users  on  a  broader  basis  is  an  issue  that 
will  be  decided  by  Mr.  Steve  Gross  and  the  Naval  Center  for 
Cost  Analysis.  There  eue  presently  no  plans  to  market  the 
SASET  model,  but  Martin  Marietta  Corporation  may  market  a 
derivative  model. 

4.  SoftCbst-Ada:  SoftCost-Ada  is  available  for  a  monthly  or 
annual  licensing  fee.  The  SoftCost-Ada  PC  version  annual 
licensing  fee  for  one  unit  costs  $8,000.  The  price  for 
additional  ccpies  is  $1,000.  A  site  license  is  $11,000.  The 
SoftCost-Ada  Vax  version  costs  $8,000  for  the  first  license 
(4  users)  and  $1,000  for  each  additional  license  (4  users). 
Prices  include  a  telefiione  help  line  and  system  upgrades  at 
no  additional  charge.  SoftCost-Ada  Vax  version  site 
licensing  agreements  are  negotiable.  A  GSA  contract  is  being 
negotiated  vhich  will  result  in  a  discount  for  DoD  users. 

5.  SPCJV'20:  A  SPQIV20  one  time  licensing  fee  for  purchasing  one 
unit  costs  $5,000.  Site  licenses  and  multi-volume, .purchcises 
are  negoticible. 

6.  SYSTHf-3:  The  System-3  annual  rate  for  government  users  is 
$9, 550/year  for  one  unit;  additional  units  (two  and  three) 
are  $800/year  and  $600/ycar  for  four  or  more  units.  The 
Systein-3  annual  licensing  fee  for  conmercial  users  is  $12,500 
for  one  unit;  additional  units  (two  ttiroucfi  four)  are 
$2, 000/year.  In  addition,  further  price  reductions  are 
available  for  blocks  of  five,  ten,  25  and  50  units.  System-3 
has  a  training  course  available.  This  course  is  strongly 
recommended  and  costs  $790/person  vhen  given  at  the  CEI 
facility.  Training  at  tlie  customer's  facility  (up  to  20 
persons  per  session)  is  $4500/session  plus  travel  and  living 
expenses  for  CEI  personnel.  Commercial  training  costs  are 
$5, 800/per  session  plus  travel  and  living  e^qsenses.  Prices 
include  a  tel^jhone  help  line  and  system  upgrades  at  no 
additional  charge. 
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SaXE  OP  OOVHOOE:  12EE  CYCIE  IBASE5  AND  ACl'lVi'llES 


With  the  exo^jtlon  of  SP^V^O,  each  model  included  in  this  Ada 
costing  stuc^  generates  an  effort  expenditure  sumnary  in  terms  of  the 
software  cost  elements  enccnpassed  by  the  estimate  and  by  life  cycle 
phase.  SPQEV'20  provides  estimates  only  in  terms  of  the  software 
project  activity.  Hcwever,  these  activities  can  be  me^pped  to  life- 
cycle  phases.  Table  E-2  shews  the  range  of  life-cycle  phases  covered 
by  each  model.  Ihases  are  mapped  to  technical  reviews  and  audits  [DO)- 
Sro-2167A]  in  order  to  provide  a  basis  for  ocrparisoi  of  models  in 
terms  of  life-cycle  coverage.  Blocks  depicted  in  Table  E-1  are 
labelled  using  the  same  phase  terminology  prescribed  by  each  model.  It 
is  evident  from  the  Table  that  phases  are  not  defined  in  a  standard  way 
across  all  models.  All  of  the  models  cover  the  operational  or 
maintenance  phase  in  additioi  to  development.  Cperaticxial  support, 
following  suooessful  ccnpletion  of  a  software  acceptance  review  (F^) , 
estimated  for  each  model  is  provided  in  Table  E-1. 

A  breakdown  of  software  cost  elements  encorpassed  by  the  model 
estimates  is  provided  in  Table  E-2.  Activities  are  described  using  the 
exact  terminology  of  the  model.  A  s^seirate  estimate  given  in  terms  of 
person-months  of  effort  is  provided  for  each  cost  element. 


TABIE  E-1 

OHSATIGNAL  SOFPCRT  AClTVmES 


CX)SIMDDL: 

Annual  maintenance 

FKECE  S; 

OperatioDcd.  support  for  user-specified 
length 

SASBT: 

Cperational  support  for  usei>-specified 
length 

SoftCost-Ala: 

Cperational  support  for  user-specified 
length 

SPQFV20: 

Ip  to  5  years  of  operatioral  svpport 

SySTEM-3: 

15  years  of  cperational  support 
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SCOPE  OF  COVERAGE:  LIFE  CTCLE  PHASES 
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table  E-3 


SCffTNARE  GOST  EIEMENIS  ENOCMEASSQ)  EV  HXEL  E5TIMA3ES 


MDDEL 

ACTIVITY 

ODSmCL 

.  Requirements  Analyses 
.  PrxDduc±  Design 
.  Programming 
.  Test  Planning 
.  Verification  and  Validation 
.  Project  Office 

.  Configuration  Management/&Jality 
Assurance 
.  Manuals 

PRICE  S 

.  Software  Design 
.  Prograimtiing 
.  Documentation 

.  Systems  Engineering  and  Program 
Management 
.  Quality  Assurance 
.  Configxnration  Management 

SASET 

.  Software  Engineering 
.  Systems  Engineering 
.  Quality  Assurance 
.  Test  Engineering 

SoftCost-Ma 

.  Software  Develcpnent 
.  SoftwEure  Management 
.  Software  Configuration  Management 
.  Software  Quedity  Evaluation 

SPQEV'20 

.  Planning 
.  Rec^iirements 
.  Design 
.  Coding 

.  IntegratioryTest 
.  Documentation 
.  Management 

SYSTEM-3 

.  Systems  Engineering 
.  Project  Management 
.  Design 
.  Programmers 
.  Qucility  Assurarxe 
.  Configuration  Management 
.  Test 

.  Data  Manipulation 
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Detailed  Description  of  tlie  hda.  axXND  Model 

Appendix  F  provides  a  description  of  the  Ada  OXXac  Model  vtiiich 
was  introduced  by  Dr.  Boehm  at  the  1987  OOOCMD  User's  Grxx?>  Conference. 
This  description  includes  the  enhanoannents  to  the  Ada  model,  namely  the 
inUirpolation  of  AGAP,  PCAP,  and  MODP  mtiltipliers,  that  Dr.  Boehm 
introduced  at  the  1988  OOCXMD  User's  Group  Conference.  Note  that 
current  estimating  equations  oily  ^jply  to  entoedded-mode  software 
projects  (see  pages  78-83,  Software  BncfTn<a*»rina  Eoonctnics) .  Equations 
have  not  been  developed  for  organic  or  semidetached  modes,  or  Basic 
OOOCMD  [B0EH88].  Ihis  overview  is  organized  into  the  follcwing 
sections: 

I.  Estimating  Nominal  Effort 

A.  Accessing  Ocnpliance  with  the  Ada  Process  Model 

B.  Ada  Instructicxi  Oounting  Rules 

II.  Adjxast  Nominal  Effort 

A.  Rate  Software  Development  Effort  Multipliers 

B.  Interpolate  ACAP,  PCAP  and  MODP  Multipliers 

C.  Apply  Effort  Multiplier 

III .  Sample  Prci>lem 


I.  Estimating  Nominal  Effort 

An  Ada  OOOCMD  software  develc^ment  effort  estimate  begins  by 
determining  the  size  of  the  effort  and  the  extent  to  which  the 
development  effort  complies  with  the  Ada  Process  Model.  Once  these 
values  have  been  determined,  a  ncmined.  effort  estimate  is  generated, 
using  the  follcwing  equation: 

MMncm  =  2.8  (KDSI)  ^  (1) 

vhere 

KDSI  =  Thousands  of  delivered  source  instructions 

b  =  extent  to  vhich  the  development  effort 
coirplies  with  the  Ada  Process  Model. 

The  values  that  b  can  assume  range  from  b  =  0,  for  full  use  of  the  Ada 
Process  Model,  to  b  =  0.20,  for  non-use  of  the  Ada  Prcxess  Model. 
Since  the  extent  of  cxmpliance  with  the  Ada  Process  Model  appears  in 
the  equation  as  an  exponent,  it  is  very  inportant  to  make  intelligent 
selections  for  each  of  the  Ada  Process  Model  cxjrpliance 
character  istic:s . 
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Assessing  Oarplianoe  with  the  Ada  Prooess  Model 

Table  F-1  describes  the  characteristics  vhich  detenidne  the  extent 
of  ccnplianoe  with  the  Ada  Process  Model  (AIM) . 

T3UaLE  F-1 

OlARACIBaSIICS  aSiKr  EETBSOME  EXTEMT  OF  (XHFUANCE  WIIH  AIM 
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1 
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1 

1 
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■ 

The  sum  of  the  individual  ratings,  b,  is  used  in  the  exponent  porticai 
of  the  ncminal  effort  equation  (1) .  Tables  F-2,  F-3,  and  F-4  are 
provided  to  aid  in  this  selection  process  for  determining  Design 
Thoroui^iness  at  FDR,  Risk  Elimination  by  FDR,  and  Requirements 
Volatility,  To  select  the  appropriate  Ada  Prooess  Model  cotplianoe 
rating,  one  averages  the  rates  of  the  respective  sipport  criteria. 
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AOA  COCONO  b  FACTOR  :  RISK  ELINIMATIOH  BY  PDR 
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arfa  TnstructiCTi  Counting  Rules 

In  order  to  ccnplete  the  nonined  effort  estimate  it  is  necesseiry 
to  determine  the  number  of  Ada  instructions  present  in  the  softwcore 
develcpnent  effort.  The  follcxwing  criteria  are  vised  to  determine  what 
an  Ada  instruction  is: 

1)  Delivered  Source  Instructions  = 

Number  of  Carriage  Returns  in  Package  Specs 
+  Number  of  Semicolons  in  Package  Bodies 

.  2)  Comments  eire  not  counted 

3)  Rajse  and  multiply-used  packages  are  counted  cis  follows: 

a)  unmodified  utility  software  is  not  counted 

b)  Count  each  invocation  statement  (  e.g. ,  WITH,  USE  ) 

c)  Use  the  Reuse  Model  to  count  pacJ^es  invoked: 

EDSI  =  (ADSI)  [0.4  (CM)  +  0.3  (CM)  +  0.3  (IM)  ]  /  100 

EDSI  =  Equivalent  new  DSI 
ADSI  =  Adapted  or  reused  instructions 
CM  =  %  of  adapted  software  design  modified 
CM  =  %  of  adapted  code  modified 
IM  =  %  of  adapted  software's  original  integration 
required  in  new  context. 


II.  Adjust  Ncminal  Effort 

Once  a  noninal  effort  estimate  (MMncm)  been  generated,  it  is 
adjusted  by  applying  effort  multipliers  determined  from  the  project's 
ratings  with  respect  to  18  cost  driver  attributes.  The  product  of  all 
18  cost  driver  attributes,  EM,  and  the  nomineil  effort  estimate,  M^^^c3m' 
yield  man-months  for  develcpment. 


18 

MM  =  MMnoni  *  tt  (EM)  (2) 

i=l 

Rate  Software  Development  Effort  fftiltipliers 

The  numerical  values  of  the  effort  multipliers  (EM)  cue  listed  in 
Table  F-5.  The  corresponding  rating  scale  is  summeurized  for  each 
effort  multiplier: 
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1)  Required  Software  Reliability  (RELSf) : 


VL  =  Error  effect,  sli^t  inconvenience 
L  =  Lew,  easily  recoverable  losses 
NCM  =  Moderate,  recoverable  losses 
H  =  Hi^  financial  loss 

VH  =  Risk  to  human  life 

2)  Data  Base  Size  (DftlA) : 

L  =  DB  bytes/program  DSI  <  10 

NCM  =  10  <=  D/P  <  100 
H  =  100  <=  D/P  <  1,000 
VH  =  D/P  >=  1,000 

3)  Software  Product  Carplexity  (OPEX): 

See  Table  8-4  in  Software  Engineering  Econatiics. 
page  122. 

4)  Required  reusability  (HBE) .  Ratings  are  eissigned  according  to  the 
extent  that  the  developer  must  design,  document,  and  test  reusable  Ada 
conponents: 

NOM  =  Ada  components  not  for  reuse  elses^iiere 
H  =  Reuse  within  single-mission  products 
VH  =  Reuse  across  single  product  line 
EH  =  Reused  in  any  afplication 

5)  Execution  time  constraint  (TIME) : 

NOM  =  <=  50  %  use  of  available  execution  time 

H  =  70  %  use 

VH  =  85  %  use 

EH  =  95  %  use 

6)  Main  storage  constraint  (STTOR) ; 

NOM  =  <=  50  %  use  of  available  execution  time 

H  =  70  %  use 

VH  =  85  %  use 

EH  =  95  %  use 

7)  Virtual  machine  volatility  for  host  cenputer  (VMVH) .  Ratings  are 
defined  in  terms  of  the  relative  frequency  of  major  changes  and  minor 
changes  to  the  underlying  virtual  machine  vhere 

o  A  major  change  significcintly  affects  roughly  10  %  of 
routines  uixier  development. 
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o  A  minor  change  significantly  affects  roughly  1  %  of 
routines  under  develciment. 

Ratings  are  assigned  as  follcxi^: 

L  =  Major;  12  months;  Minor:  1  month 

NCM  =  Major:  6  months;  Mirwr;  2  weeks 

H  =  Major:  2  months;  Minor:  1  week 

VH  =  Major;  2  weeks;  Minor:  2  days 

8)  Virtual  machine  volatility  for  target  ccnputer  (VMVT) .  The  same 
definition  for  host  effects  applies  to  target  effects. 

Ratings  are  eissigned  ats  follows: 

L  =  Major;  12  months;  Minor:  1  month 

NCM  =  Major:  6  months;  Minor;  2  weeks 

H  =  Major:  2  months;  Minor:  1  week 

VH  =  Major:  2  weeks;  Minor:  2  days 

9)  Ccmputer  turnaround  time  (TOEN) .  Ratings  are  defined  in  terms  c 
average  response  time  in  hours  (frcm  the  time  the  developer  sufcndts 
jcfc  to  be  run  until  the  results  ctre  back  in  the  develcper's  hands) ; 

VL  =  Interactive,  1  terminal/person,  full  lifecycle 
support 

L  =  Interactive,  0.3  terminals/person,  focused  on  code, 
test  support 

NCM  =  <  4  hours  turnaround  time 
H  =4-12  hours  turraround  time 
VH  =  >  12  hours  turnaround  time 

10)  Analyst  capability  (ACAP) : 

VL  =  15th  percentile  team 
L  =  35th  percentile  team 
NCM  =  65th  percentile  team 
H  =  75th  percentile  team 
VH  =  90th  percentile  team 

11)  Programmer  capability  (PCAP) ; 

VL  =  15th  percentile  team 
L  =  35th  percentile  team 
NCM  =  65th  percentile  team 
H  =  75th  percentile  team 
VH  =  90th  percentile  team 
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to 


12)  Applications  experience  (AEXP) ; 


VL  =  <=  4  months  average  experience 
L  *1  year  average  experience 
NCM  *  3  years  average  experience 
H  =6  years  average  experience 
VH  =  >=  12  years  average  experience 

13)  Virtual  machine  experience  (VEXP) ; 

VL  =  <=  1  month  average  experience 
L  =4  mcnths  average  experience 
NCM  =  1  year  average  experience 
H  =  >=  3  years  average  ejqjerienoe 

14)  Progranming  language  ejq^erience  (I£XP) .  Duration  of  experience  of 
the  project  team  developing  the  software  module  with  the  programming 
language  to  be  used: 

VL  =  <=  1  month  average  experience 
L  =4  months  average  experience 
NCM  =  1  year  average  ejperience 
H  =3  years  average  experience 
VH  s=  >=  6  years  average  experience 

15)  Use  of  modem  programming  developing  practices  (MXJP) : 

VL  =  No  use  MODPs 
L  =  Beginning  use 
MCM  =  Scare  use 
H  =  General  use 
VH  =  Routine  use 

16)  Use  of  software  tools  (TOtJL) .  Ratings  are  assigned  according  to 
the  degree  to  vAiich  software  tools  cure  used  in  developing  the  software 
subsystem: 

VL  =  Very  few,  primitive  tools 
L  =  Basic  micro  tools 
NCM  =  Basic  mini  tools 
H  =  Basic  maxi  tools 
VH  =  Extensive  tools,  little  integration 
EH  =  Moderately  integrated  environment  (UNIX) ;  strong 
target  tool  svpport 

XXH  =  Fully  integrated  environment;  rating  not  acheived 
by  any  support  environment  to  date,  rating  can  be 
dropped  for  neeu:  term;  strong  target  tool  sipport 
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17)  Required  developnent  sciiedule  (SCED) : 


VL  =  75  %  of  nondral  schedule 

L  =  85  % 

NCM  =  100  % 

H  =  130  % 

VH  =  160  % 

18)  Classified  security  applicatioi  (SECD) .  Ratings  are  assigned 
accourding  to  security  cind  privacy  restrictions: 

NCM  =  Unclassified 
H  =  Classified  (Secret,  Tcp  Secret) 


InterpolatiOTi  of  ACAP.  PCAP.  and  tCTP  Effort  Multipliers 

Seme  of  the  effort  multipliers  are  affected  by  the  extent  of  Ada 
Process  Model  cenplianoe:  ACAP,  PCAP,  and  MODP.  To  determine  vhat  the 
value  of  these  modified  effort  multipliers  should  be,  one  must 
interpolate  between  numerical  valiies  for  the  full  and  non-use  cases  of 
Ada  Process  Model  oenplianoe.  The  ratio,  b  /  0.16,  provides  a  measure 
to  determine  the  distance  between  the  value  for  the  effort  multiplier 
for  Ada  OOOCMD  and  standard  CXXXMD.  In  the  following  exanple  (Exhibit 
F-2) ,  0.08  is  the  value  used  for  b,  ACAP  is  set  to  lew,  PCAP  to  very 
lew,  and  MODP  to  nominal.  The  effect  of  the  interpolation  of  these 
effort  multipliers  is  to  extend  their  range. 
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ADA  axnO  b  FACTOR:  INIEREREJATICN  EFFECTS  [BMH8aA] 
For  Effort  Multipliers  (EMs)  ACAP,  PCAP,  and  MODP 
Hew  =  ^full  +  b/0-16  (  EMncn  -  EMfun) 


COST 

DRIVER 

VERY 

DOW 

DOW 

NOMINAL 

HIGH 

VERY 

HIGH 

ACAP 

1.57  - 
1.46 

1.29  - 
1.19 

1.00  - 
1.00 

0.80  - 
0.86 

0.61  - 
0.71 

PCAP 

1.30  - 
1.42 

1.12  - 
1.17 

1.00  - 
1.00 

0.89  - 
0.86 

0.80  - 
0.70 

MODP 

1.10  - 
1.10 

0.98  - 
1.00 

0.86  - 
0.91 

0.78  - 
0.82 

ACAPnev,= 

1.29 

+ 

0.08  /  0.16 

( 

1.19  -  1.29  )  =  1.24 

rcAPnew  = 

1.30 

+ 

0.08  /  0.16 

( 

1.42  -  1.30  )  =  1.36 

M0DPnew  = 

0.98 

+ 

0.08  /  0.16 

( 

1.00  -  0.98  )  =  0.99 

Apply  Effort  Multiplier 

Once  the  interpolation  has  been  performed,  the  product  of  the  18 
life  and  the  ncniinal  effort  estimate,  are  used  to  adjust  naniinal 

effort  (2) .  Once  the  adjvisted  affort  is  known  (MM) ,  the  develcpnent 
schedule  can  be  estimated.  The  following  equation  is  eitployed  to 
ccnpute  the  development  schedule  estimate: 

Tdev  =  3.0  (MM)  0*32  +  (0.2  *  b) 
vAiere 

"^dev  ~  Time  in  months  from  Software  Requirements 

Review  (SRR)  to  Software  acc^jtance  test  (FQT) 


III.  Sanple  Prctolem 

Let  us  run  throu<^  an  exaiiple,  using  one  of  the  projects  targeted 
in  the  test  case  study,  Project  6.  We  start  with  the  selections  for 
extent  of  cotnplicince  with  the  Ada  Process  Model.  The  characteristic 
being  evaluated,  its  rating  and  associated  numerical  value  are  provided 
in  Table  F-7. 
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ADA  PROCESS  MOCEL  CHARAdEKISnCS  FOR  SAMPLE 
CHARACIEKISnC  RAmiG  VALUE 


Experience  with  the  ATM 

Seme  e^qserience 

0.03 

Design  thorouj^Tness  by  PtR 

60  %  eliminated 

0.03 

Risks  Eliminated  by  PCR 

90  %  eliminated 

0.01 

Requirements  Volatility 
During  Development 

Frequent  moderate 
changes 

0.04 
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Ihe  values  of  the  ratings  are  sunmed: 

b  =  0.03  +  0.03  +  0.01  +  0.04  =  0.11 

There  were  24  K  cJelivered  source  instructions  (DSI) ,  so  a  nonincil 
effort  estimate  may  be  ccnputed  as  follows: 

MMncm  =  2.8  (KDSI)  ^  =  2.8  (24)  =  108.24 

Since  b  =  0.11  r^resents  partiail  ccnpliance  with  the  Ma  Process 
Model,  we  interpolate  to  arrive  at  new  effort  multipliers  for  ACAP, 
PCAP,  and  MM)P  as  below  (  and  in  Ebdiibit  F-4) : 


^^^new  “ 

0.80 

+ 

0.11  /  0.16 

( 

0.86  -  0.80  )  ^  0.84125 

PCAPnew  = 

0.89 

+ 

0.11  /  0.16 

( 

0.86  -  0.89  )  =  0.869375 

M0DPnew  = 

0.98 

+ 

0.11  /  0.16 

( 

1.00  -  0.98  )  =  0.99375 

TftBIE  F-« 

INIEKPOIATES  VAIUE5  FtR  SAMFI£  FHBEfM 


Effort 

Miltiplier 

Rating 

/alue 

Ada 

■ 

Interpolated 

Value 

ACAP 

H 

— 

0.86 

0.80 

0.11 

0.84125 

PCAP 

H 

0.86 

0.89 

0.11 

0.869375 

MXP 

NCM 

1.00 

0.98 

0.11 

0.99375 

Values  for  the  effort  multipliers  are  as  follows: 


T?Rr.Y 

Value 

DATA 

Value 

H 

1.07 

NCM 

1.00 

TIME 

Value 

STGR 

Value 

H 

1.11 

NCM 

1.00 

TOFN 

Value 

ACAP 

Value 

VL 

0.79 

H 

0.84125 

CPIX 

Value 

FUSE 

Value 

H 

1.08 

H 

1.10 

VMVH 

Value 

VMWT 

Vcilue 

NCM 

1.00 

L 

0.93 

PCAP 

Vcilue 

AEXP 

Value 

H 

0.869375 

H 

0.91 
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VEXP  Value  lEXP  Value  MXP  Value  TOOL  Value 

VL  1.21  VL  1.26  NCM  0.99375  NCM  1.00 

SCH3  Value  SEX3J  Value 

NCM  1.00  NCM  1.00 


Ihe  man-months  for  develcpnent,  Ml,  can  now  be  ocai^xited  using  the 
effort  equation: 

18 

IM  =  ^ 

i=l  i 

Substituting,  (M  is  calculated: 

Ml  =  108.24  *  1.045  =  113.14 

With  man-months  for  development,  MM,  we  can  new  generate  the  schedule 
estimate  using  the  following  relationship: 

Tdev  =  3.0  (113.14)  0*32  +  (0.2  *  b) 

=  3.0  (113.14)  0*342  =  15,12 

So,  for  this  project,  effort  multipliers  extended  the  man-months 
for  develcpnent  from  the  ncminal  estiitate  of  108.24  to  113.14  man- 
months.  The  development  schedule  estimate  was  ccsrrputed  to  be  15.12 
months. 
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Project  Questicmcdre 


This  project  questionnaire  contains  the  cSata  items  that  are 
required  to  run  the  follcwing  software  cost  and  schedule  estimating 
models: 


o 

Ada  CXXXMD 

o 

EKECE  S 

o 

SoftOost-Ada 

o 

ESTIMACS 

o 

SYSTEM-3 

o 

SPC2IV'20 

o 

SASET 

An  abbreviated  version  of  this  form  was  used  to  collect  information  on 
six  of  the  nine  projects  targeted  in  the  test  case  study.  The  form  is 
itiodul2u:  and  is  divided  into  the  following  ei^t  sections: 

1.  General  Information 

2 .  Project  Description 

3.  Development  Methodology 

4 .  Software  Size 

5.  Project  Staffing 

6 .  Corputer  System 

7 .  Development  Environment 

8.  Resource  Allcxxition. 
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PROJECT  QUESTIONNAIRE 


GENERAL  INFORMATION 

Please  complete  this  form  to  the  best  of  your  ability  for  your 
project.  If  the  question  is  not  applicable,  please  mark  it  N/A. 
If  you  don't  know  the  answer,  leave  it  blank.  Mark  each  page 
containing  confidential  or  proprietary  data  "CONFIDENTIAL"  on 
both  its  top  and  bottom  in  bold  letters. 

1.  Your  name: _  Date ;  /  / 

2.  Title: _  Phone:  (  )  _ 

3.  Firm  or  organization: _ _ 

Address : _  _ 


4.  Name  of  project: _ 

5.  Contract  number: _ 

6.  Customer  name: _ 

7.  Project  overview  description: 


I 

I 

I 

I 


8.  Developer  contact; _  Phone:  i _ )_ 

9.  Customer  contact: _  Phone:  ^ _ 1 

10.  Current  status:  _ 
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PROJECT  QUESTIONNAIRE 


PROJECT  DESCRIPTION 


1 .  System/Software  Characteristics 

a.  Application  Type  (check  all  that  apply) : 

[  ]  Prototype  to  be  discarded  later 
[  ]  Prototype  to  be  built  into  delivered  program 
[  ]  Complete  stand-alone  program 
[  ]  Component  within  a  system 
[  ]  Reusable  component  for  multiple  programs 
[  ]  Release  with  new  features 
[  ]  Release  with  defect  repairs 
[  ]  System  with  multiple  components 

b.  Project  Class  (check  all  that  apply) : 


c 

] 

Used  at  single  location 

[ 

] 

Military  contract 

[ 

] 

Used  at  multiple  locations 

[ 

] 

Government  contract 

[ 

] 

Sold  as  retail  product 

[ 

] 

Commercial  contract 

[ 

] 

For  public  domain 

[ 

] 

Leased  to  users 

[ 

] 

Bundled  with  hardware 

[ 

] 

For  personal  use 

[  ]  Developed  in  an  Academic  Environment 
[  ]  Used  remotely  via  time-sharing 
[  ]  Using  military  specifications 

c.  Applications  domain  (check  all  that  are  applicable) : 


[ 

] 

Automation 

[  ] 

Data  Processing 

[ 

] 

Business 

[  ] 

Environments/Tools 

[ 

] 

Command  &  Control 

[  ] 

Production  Control 

[ 

] 

Communications 

[  ] 

Support  Software 

[ 

] 

Signal  Processing 

[  ] 

Process  Control 

[ 

] 

Test  Systems 

[  ] 

Scientific 

[ 

] 

Trainers 

[  ] 

Robotics/Mechanical 

[ 

] 

Avionics 

[  ] 

Unmanned  Flight 

[ 

] 

Interface  Systems 

[  ] 

Manned  Flight 

[ 

] 

Graphics,  image  processing 

[ 

] 

Other 

d.  Software  Interface  Complexity;  How  many  software  systems 
and  peripheral  communications  equipment  does  this  software 
system  interface  with?  _ 
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PROJECT  QUESTIONNAIRE 


PROJECT  DESCRIPTION,  cont'd. 

e.  Rate  the  difficulty  of  processing  logic: 

[  ]  Simple  processing  logic,  straight-forward  I/O 
[  ]  Difficult  highly  nested  logic,  real-time  processing 
[  ]  Routine  nesting,  minimal  interface  with  Operating 
system,  standard  I/O 

[  ]  Complex  dynamic  resource  allocation,  multiple 
exception  handlers,  recursion 

f.  Product  Structural  Complexity 
[  ]  Nonprocedural 

[  ]  Well  structured  reusable  modules 
[  ]  Small  modules  and  simple  paths 
[  ]  Complex  modules  and  paths 
[  ]  Large  modules  and  complex  paths 

g.  Mathematical  Complexity 

[  ]  Simple  algorithms  and  simple  calculations 
[  ]  Majority  of  simple  algorithms  and  calculations 
[  ]  Algorithms  and  calculations  of  average  complexity 
[  ]  Some  difficult  or  complex  calculations 
[  ]  Many  difficult  algorithms  and  complex  calculations 

h.  Degree  of  Real-Time 

[  ]  No  tasking;  essentially  batch  response 
[  ]  Interactive  with  limited  Ada  tasking 
[  ]  Interrupt  driven  with  tasking  in  milliseconds 
[  ]  Concurrent  tasking  with  rendezvous  in  milliseconds 
[  ]  Concurrent  tasking  with  rendezvous  in  nanoseconds 

i.  List  the  percent  of  total  source  code  allocated  to  each  of 
the  following  operational  timing  requirements; 

Real-Time  %  On-line 

Time-Constrained  _ %  Non-Time-Critical 


j .  Select  the  percent  of  source  code  with  real-time 
considerations : 

[  ]  0%  [  ]  25%  [  ]  50%  [1  100% 

2 .  Database  Characteristics 

a.  Number  of  physical  data  files: _ 
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PROJECT  QUESTIONNAIRE 
PROJECT  DESCRIPTION,  cont'd. 
b.  Database  Complexity 

[  ]  simple  data,  few  variables,  low  complexity 
[  ]  Simple,  numerous  variables 
[  ]  Multiple  files,  fields  data  interactions 
[  ]  Complex  file  structure 
[  ]  Highly  complex 

3.  Reliability 

a.  Effect  of  a  software  failure 

[  ]  Inconvenience  [  ]  Moderate  loss 

[  ]  Easily-recoverable  loss  [  ]  Major  financial  loss 
[  i  Loss  of  human  life 

b.  Backup/Recovery  Considerations 

[  ]  Data  protection  beyond  regular  backup  required 
[  ]  Alternative  methods  need  to  be  developed  in  case 
of  software  failure 
[  ]  No  special  backup  requirements 

4 .  User  Interface 

a.  Display  Requirements 
[  ]  Simple  I/O 

[  ]  User-friendly,  menu  driven 

[  ]  Pressure  sensitive  devices  (touch  screen,  joystick) 
[  j  Graphics  oriented 

5.  Software  Testability 

[  ]  Very  difficult  [  ]  Time  insensitive 

[  ]  Difficult  [  ]  Easy 

6.  Is  this  the  first  system  of  its  kind? 

[  ]  Yes  [  3  No 
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PROJECT  QUESTIONNAIRE 


PROJECT  DESCRIPTION,  ContM. 


7 .  Reused  Code 

a.  Logical  Complexity  of  Reused  Code: 

[  ]  Simple  algorithms  an  dsimple  calculations 
[  ]  Majority  of  simple  algorithms  and  calculations 
[  ]  Algorithms  and  calculations  of  average  complexity 
[  ]  Some  difficult  or  complex  calculations 
[  ]  Many  difficult  algorithms  and  complex  calculations 

b.  Structural  Complexity  of  Reused  Code: 

[  ]  Nonporcedural  (generated,  query,  spreadsheets,  etc.) 

[  ]  Well  structured  with  reusable  modules 
[  ]  Well  structured  (small  modules  and  simple  paths) 

[  ]  Fair  structure  but  some  complex  paths  and  modules 
[  ]  Poor  structure  with  many  complex  paths  and  modules 

c .  Database  Complexity 

[  ]  Simple  data  with  few  variables  and  little  complexity 
[  ]  Several  data  elements  byt  simple  data  relationships 
[  ]  Multiple  files,  switches,  and  data  interactions 
[  ]  Complex  data  elements  and  domplex  data  interactins 
[  ]  Very  complex  data  elements  and  data  interactions 

d.  Select  the  intended  use  of  the  majority  of  the  software 
packaged  for  reuse: 

[  ]  None 

[  ]  Reuse  within  single  mission  products 
[  ]  Reuse  across  single  product  line 
[  ]  Reuse  in  any  application 

8 .  General 

a.  Scope  of  Support 

[  ]  No  support  to  non-software  organizatoins 
[  3  Liaison  support  to  non-software  organizations 
[  ]  Extensive  support  to  system  test  organizations 
[  ]  Extensive  support  to  system  entineering  and  test 
organizations.  CSSR/CSCSC  reporting  requirements. 


PROJECT  QUESTIONNAIRE 


nRVRT/^PMENT  METHODOLOGY 
1, Milestones 

Enter  the  expected  and  actual  dates  for  each  milestone  below, 
or  N/A  if  the  milestone  does  not  apply  to  this  project.  If  an 
expected  date  is  an  estimated  date  rather  than  a  contract 
date,  put  an  asterisk  after  that  date. 

Expected  Actual 

Milestone  Date  Date 


Project,  Start  Date  /  /  /  / 


System  Requirements  Review  (SRR) 

/ 

/ 

/ 

( 

Software  Specification  Review  (SSR) 

/ 

/ 

/ 

/ 

System  Design  Review  (SDR) 

/ 

/ 

/ 

( 

System  Hardware  Preliminary  Design  Review 

/ 

/ 

/ 

/ 

System  Software  Preliminary  Design  Review 

( 

/ 

/ 

/ 

System  Software  Critical  Design  Review 

( 

/ 

/ 

( 

Test  Readiness  Review  (TRR) 

/ 

/ 

/ 

/ 

Functional  Configuration  Audit  (FCA) 

/ 

/ 

/ 

( 

Physical  Configuration  Audit  (PCA) 

( 

/ 

/ 

/ 

Formal  Qualification  Review  (FQR) 

/ 

/ 

/ 

( 

Operational  Test  and  Evaluation  (OTE) 

/ 

/ 

/  . 

( 

Project  Completion  Date 

-J-. 

Z  - 

-J- 

.  Z  - 

2 .  Schedule 

a.  Select  the  schedule  and  staffing  constraints  which  best 
describe  this  development: 

[  ]  Normal  average  schedule,  effort,  and  quality 
[  ]  Shortest  development  schedule,  extra  staffing 
[  ]  Lowest  cost  with  reduced  staffing 
[  ]  Highest  quality  and  reliability 

[  ]  Shortest  schedule  with  high  quality  and  reliability 

[  ]  Match  staff  size,  with  normal  development 

L  J  Match  staff  size,  with  shortest  schedule 

[  ]  Match  staff  size,  with  very  high  quality 

b.  Schedule  Aggressiveness  (as  %  of  nominal  effort;  i.e.,  if 
schedule  is  extremely  curtailed,  aggressiveness  would  be 
greater  than  160%) 

[  ]  75% 

[  ]  100%  (normal  development  schedule) 

[  ]  130% 

[  ]  greater  than  160% 
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PROJECT  QUESTIONNAIRE 


nKVEIiOPMENT  METHODOLOGY 


3 .  Development  Standards 

a.  Check  all  types  of  standard  used  in  this  development: 

[  ]  Commercial  [  ]  IEEE  [  ]  Military 

[  ]  Other  [  ]  None 

b.  List  the  name(s)  of  these  standard(s): 


c.  Were  these  standards  tailored  specifically  for  use  on  this 
effort? 


[  ]  Yes  [  ]  No 

d.  Do  you  have  a  set  of  database  standards  and  administrative 
procedures  that  were  used  in  this  development? 

[  ]  Yes  [  ]  No 

4.  Development  paradigm  employed  (check  all  that  apply): 

[  ]  Waterfall  development  [  ]  Incremental  development 
[  ]  Phased  builds  t  ]  Spiral  development 

[  j  Rapid  Prototyping  t  ]  Pilot  development 

[  ]  Other  _ 

5.  Software  Reviews 

a.  Select  all  informal  reviews  held  on  the  software  during 
this  development: 

[  ]  Design  walkthroughs 
[  ]  Design  inspections 
[  ]  Code  walkthroughs 
[  ]  Code  inspections 

[  ]  Other  _ 

b.  Select  all  management  reviews  held  on  the  software  for  this 
project: 

[  ]  Monthly  project  reviews 
[  ]  Weekly  status  reviews 

[  ]  Other;  _ 
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PROJECT  QDESTIONNAIRE 
DEVEIiOPMEMT  METHODOLOGY,  cont'd. 


6.  Documentation  Requirements 

a.  Number  of  documents  required:  _ 

b.  Total  number  of  pages  of  all  documents:  _ 

c.  Select  any  of  the  following  tools  used: 

[  ]  Design  document  generator 
[  ]  Graphics  generator 
[  ]  Text  automation 

[  ]  Other  _ 

d.  Documentation  is  primarily  written  by: 

[  ]  Professional  writers  [  ]  Programmers 

[  ]  Other  _ 

7 .  System/Software  Requirements 

a.  Select  the  option  which  corresponds  to  the  level  of 
definition  and  understanding  of  system  requirements: 

[  ]  Very  little  definition  and  understanding  of  system 
requirements 

[  ]  Questionable  definition  and  understanding  of  system 
requirements 

[  ]  Fairly  complete  definition  and  understanding  of 
system  requirements 

[  ]  Very  complete  definition  and  understanding  of 
system  requirements 

b.  How  will  overall  technology  changes  impact  the  project? 

[  ]  During  the  development,  the  requirements  will  change 
more  than  once  to  upgrade  the  system,  due  to 
significant  improvements  in  technology 
[  ]  During  the  development,  there  will  be  at  least  on 
(but  less  than  three)  significant  modifications  to 
the  system  due  to  technology  upgrades 
[  ]  During  the  development  there  will  be  some  minor 
modifications  due  to  technology  upgrades 
[  ]  There  will  be  no  changes  to  the  system  or 
requirements  during  the  development  effort 
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PROJECT  QUESTIONNAIRE 
DRVETOPMENT  METHODOLOGY.  COnt'd. 


c.  Rate  requirements  volatility  during  development: 
[  ]  No  changes 

[  ]  Small  non-critical  changes 
[  ]  Frequent  non-critical  changes 
[  ]  Occasional  moderate  changes 
[  ]  Frequent  moderate  changes 
, [  ]  Many  large  changes 

8.  Design 

a.  Rate  the  maturity  of  the  design  concepts  used: 

[  ]  Experimental  [  ]  Evolutionary 
[  ]  State-of-the-art  [  ]  Mature 

b.  Select  any  of  the  following  design  technologies, 
strategies,  and  tools  used: 


c 

] 

Applications  Generator 

C  ] 

Object-oriented  methods 

c 

] 

CASE  tools 

[  3 

Structured  analysis 

[ 

] 

Relational  database 

[  3 

Query 

Language  (SQL) 

[ 

] 

4GL  or  5GL 

[  3 

Reuse 

libraries 

[ 

] 

Exception  handling 

[  3 

Front 

loaded  scheduling 

[ 

] 

Management  toolset 

[  3 

Cost 

and  schedule  model: 

[ 

] 

Database  management  system 

[ 

] 

Small  up-front  design  teams 

[ 

] 

Continuous  integration 

via  package 

specifications 

[ 

] 

Ada  PDL 

[ 

] 

Other  PDL 

c.  Select  the  percentage  of  package  specifications  compiled 
successfully  at  PDR: 

[  ]  20%  [  ]  40%  [  ]  60%  [  ]  75%  [  ]  90%  [  ]  100% 

d.  Select  the  percentage  of  risks  eliminated  by  PDR: 

[  ]  20%  [  ]  40%  [  ]  60%  [  ]  75%  [  ]  90%  [  ]  100% 
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PROJECT  QUESTIONNAIRE 


DEVELOPMENT  METHODOLOGY,  cont'd. 


9.  Integration 

a.  Rate  the  expected  level  of  difficulty  of  integrating  and 
testing  the  components  to  the  CSCI  level: 

[  ]  No  internal  integration. 

[  ]  Very  little  integration,  no  complex  interfaces. 

[  ]  Average  degree  of  CSCI  integration  and 
interface  complexity. 

[  ]  Several  CSCI  interfaces  with  some  complex 
interfaces. 

[  ]  Complex,  time- intensive  CSCI  integration  process 
anticipated. 

b.  Rate  the  expected  level  of  difficulty  of  integrating  and 
testing  the  CSCIs  to  the  system  level: 

[  ]  Very  little  integration,  no  complex  interfaces. 

[  ]  Average  degree  of  system  integration  and  interface 
complexity. 

[  ]  Several  system  interfaces  with  some  complex 
interfaces . 

[  ]  Complex,  time-intensive  system  integration  process 
anticipated. 

10.  Commercial  off-the-shelf  Software  (COTS) 

a.  Select  the  option  which  best  describes  the  expected  impact 
of  integrating  commercial  off-the-shelf  software  into  the 
system: 

[  ]  There  will  be  many  impacts  on  the  design/ development 
effort  to  ensure  that  the  vendor  supplied  COTS 
software  will  interface  correctly  with  the  developed 
operational  software. 

[  ]  There  will  be  some  impacts  on  the  design/development 
effort  to  ensure  that  the  vendor  supplied  COTS 
software  will  interface  correctly  with  the  developed 
operational  software. 

[  ]  There  will  be  few  impacts  created  by  the  COTS  software 
packages  to  support  the  operating  environment  of  the 
applications  software. 

[  ]  There  will  be  no  impacts  caused  by  the  purchased 

software  as  the  purchased  software  only  performs  an 
operating  environment  support  function  (i,e., 
operating  system) . 
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PROJECT  QUESTIONNAIRE 


DEVELOPMENT  METHODOLOGY,  cont'd. 

11.  Use  of  Software  Tools 

[  ]  Basic  Ada  language  tools  [ 
[  ]  Full,  life  cycle  APSE  [ 


]  MAPSE 
3  APSE 
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PROJECT  QUESTIONNAIRE 


1.  Size  Estimates 

a.  Number  of  major  software  subsystems/programs: _ 

b.  Select  the  basis  for  size  estimates; 

[  ]  KSLOCs  [  ]  Carriage  returns 

[  ]  Function  Points  [  ]  Semicolons 

[  ]  Other : _ 

c.  Enter  the  requested  sizing  information  below,  in  thousands 
of  lines  of  source  code  (KSLOCs) . 


Deliverable 

Program 


Size  Language 

Reused  Modified 


d.  Reused  Design:  _ % 

e.  Amount  of  software  packaged  to  be  reused: 
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PRCXTECT  QUESTIONNAIRE 


SOFTWARE  SIZE 


2 .  Source  Code  Mix 

a.  CSCI  Source  Code  Mix 

Source  Code  Type  %  Code  %  New  Design  %  New  Code 

Operating  Systems . .  .  . 

Interactive  Operations . .  .  . 

Real-Time  Command  &  Control..  _  _  _ 

On-Line  Communications . .  .  . 

Data  Storage  &  Retrieval . .  .  . 

String  Manipulation . .  .  .  . 

Mathematical  Operations . .  .  . 

Other .  .  .  . 

Other .  .  . 


b.  Statement  Types 

Logical 
Data  Typing 
Ada  Tasking 
Mathematical 


3.  Function  Points 

If  the  system  was  sized  using  function  points,  please  provide 

the  following  information: 

a.  Number  of  inputs  (unique  major  data  types  that  enter  the 

system) ; _ 

b.  Number  of  outputs  (unique  logical  major  report  formats  the 

system  will  generate) : _ 

c.  Number  of  inquiries  (types  of  queries  that  result  in 

informational  searches  and  responses) : _ 

d.  Number  of  external  interfaces; _ 

e.  Number  of  internal  files  (unique  logical  files/databases 

used  by  the  application) : _ 

4.  Database  Size 

a.  Database  size,  as  a  percentage  of  total  program  size: 

% 


%  Command 

%  Declaration 

%  Invocation 

%  Data  Manipulation 
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PROJECT  QUESTIONNAIRE 


1.  Staff  Size/Availability 

a.  Total  staff: _ (staff -months  effort  end-to-end) 

b.  Minimum  staff  size: _ 

c.  Maximum  staff  size: _ 

d.  Staff  Availability: _ % 

(If  all  project  personnel  are  full-time,  enter  100%.) 

2.  Staff  Skill/Experience 

a.  Skill  Level  of  Manager 


[  ] 

Bottom  15%  [  ]  35% 

[ 

] 

55% 

[ 

] 

75% 

[ 

] 

Top 

90% 

Skill 

Level  of  Analysts 

[  ] 

Bottom  15%  [  ]  35% 

t 

] 

55% 

[ 

] 

75% 

[ 

] 

Top 

90% 

Skill 

Level  of  Programmers 

[  ] 

Bottom  15%  [  ]  35% 

[ 

] 

55% 

[ 

] 

75% 

[ 

] 

Top 

90% 

d.  Analysts'  experience  with  similar  applications:  _ years 

_ months 

e.  Select  the  option  below  that  best  describes  the  experience 
of  the  staff  on  this  project: 

[  ]  All  experts  in  the  type  of  program  being  developed 
[  ]  Majority  of  experts  but  some  new  hires  or  novices 
[  ]  Even  mixture  of  experts  and  new  hires  or  novices 
[  ]  Majority  of  new  hires  or  novices  with  few  experts 
[  ]  All  personnel  are  new  to  this  kind  of  program 

f.  Programmers'  experience  with  this  host  machine: 

_ years  _ months 

g.  Host  Machine  Expertise: 

[  ]  Completely  new  hosting  hardware  system 
[  ]  Mostly  new  hosting  hardware  system 
[  ]  Most  of  the  hardware  system  has  been  utilized  by 
members  of  the  development  team  before 
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PROJECT  QUESTIONNAIRE 
PROJECT  STAFFING,  cont'd. 

h.  Software  Language  and  Operating  System  Expertise: 

[  ]  Completely  new  hosting  hardware  system 
[  ]  Mostly  new  hosting  hardware  system 
[  ]  Most  of  the  hardware  system  has  been  utilized  by 
the  company  before 

[  ]  Extensive  experience  with  hardware  system 

i.  Analysts'  experience  with  chosen  development  Methodology: 

_ years  _ ^months 

j.  Analysts'  experience  with  the  Ada  language:  _ years 

_ months 

k.  Number  of  Ada  Projects  Completed  by  Team  Members: _ 

l.  Analysts'  experience  with  Ada  Process  Model: 

[  ]  No  familiarity  with  practices 

[  ]  Little  familiarity  with  practice 

[  ]  Some  familiarity  with  practices 

[  ]  General  familiarity  with  practices 

[  ]  Successful  on  1  mission  critical  project 

[  ]  Successful  on  at  least  1  mission  critical  project 

m.  Analysts'  experience  with  Modern  Programming  Development 
Practices  (MODP) 

[  ]  No  Use 
[  ]  Beginning 

[  ]  Reasonably  Experienced  in  some  MODP 
[  ]  Reasonably  Experienced  in  most  MODP 
[  ]  Routine  Use  of  all  MODP 

n.  Programmers'  Ada  Environment  Experience 


[ 

] 

Less  than  3 

months  of  experience 

[ 

] 

Between  3  - 

6  months  of  experience 

[ 

] 

Between  6  - 

12  months  of  experience 

[ 

] 

Over  1  year 

of  experience 
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PROJECT  QUESTIONNAIRE 


PROJECT  STAFFING.  COnt'd. 

o.  Company's  experience  developing  this  type  of  application 
[  ]  This  application  is  a  new  project  not  in  our 
current  line  of  business 

[  ]  This  application  is  a  normal  development  project 
that  is  part  of  our  current  line  of  business 
[  ]  This  application  is  a  familiar  type  of  project 

having  already  been  developed  by  the  company  before 
or  similar  to  other  projects  we  have  developed 
[,  ]  Many  applications  of  this  type  have  been  developed 
by  the  company  (greater  than  7) 

3 .  General 

a.  Staffing  will  be  done  by 

[  ]  Developer 

[  ]  Developer  and  user 

b.  Enter  the  average  work  week  for  the  staff  on  this  project: 
_  hours 

c.  Enter  the  normal  work  year  within  your  company:  _  days 

4.  Teamwork  Capability 

a.  Select  the  option  which  best  represents  the  project 
analysts'  performance  as  a  team: 

[  ]  Non-Functional  Team 

[  ]  Functional  but  not  very  effective 

[  ]  Functional  and  Effective 

[  ]  Extraordinary 

[  ]  Near  Perfect  Functioning 

b.  Select  the  option  which  best  represents  how  well  the 
programmers  working  on  the  project  will  perfoim  as  a  team: 


[ 

] 

Non-Functional 

Team 

[ 

] 

Functional  but 

not  very  effective 

[ 

] 

Functional  and 

Effective 

[ 

] 

Extraordinary 

[ 

] 

Near  Perfect  Functioning 

c.  Select  the  type  of  team  used  for  software  development 


[  ]  Design  teams  (  ]  Programming  teams 

[  ]  Interdisciplinary  teams  [  ]  Participatory  teams 


PROJECT  QUESTIONNAIRE 


1 .  Development  Environment 

a.  Configuration 
[  ]  PC  based 

[  ]  Mainframe (s)  with  ASCII  terminals 
[  ]  Mainframe (s)  with  smart  terminals 
,  [  ]  Mainframe (s)  with  PC  workstations 
[  ]  Mainframe (s)  with  minus  networked  to  workstations 
[  ]  Mainframe (s)  as  part  of  networked  homogeneous 
environment 

[  ]  Mainframe'  ‘  as  part  of  networked  heterogeneous 
environme  c 
[  ]  Other: _ 

b.  Number  of  different  types  of  workstation:  _ (0  to  100) 

c.  Rate  the  virtual  machine  volatility  of  the  development 
system,  based  on  frequency  of  major/minor  changes: 

[  ]  12  months  (major)/l  month  (minor) 

[  ]  6  months/ 2  weeks 

[  ]  2  months/ 1  week 

[  ]  2  weeks/ 2  days 

d.  Select  the  following  option  that  best  assesses  the  embedded 
features  of  the  development  system: 

[  ]  Hardware  is  to  be  developed,  but  its  completion 

will  occur  long  before  the  software  is  to  be  ready 
[  ]  Hardware  is  to  be  developed  on  the  contract,  it  is 
to  be  developed  concurrently  with  the  software  and 
the  hardware  can/does  have  major  impacts  on  the 
software 

[  ]  Hardware  is  to  be  developed  on  the  contract,  it  is 
to  be  developed  concurrently  with  the  software  but 
the  hardware  has  little  impact  on  the  software 
[  ]  No  new  hardware  is  to  be  developed  under  the 

effort;  there  will  be  no  impact  on  the  software 
development 
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PROJECT  QUESTIONNAIRE 


COMPUTER  SYSTEM 


e.  Rate  the  software  tool/environment  stability  of  the 
development  system: 


[  ]  Buggy  compiler.  APSE  change  every  2  weeks. 

[  ]  Stable  but  incapable  compiler.  APSE  change  every 
month.  New  tool  rate  1  per  week. 

[  ]  Stable  compiler.  APSE  change  every  3  months.  New 
tool  rate  1  per  quarter. 

[  ]  Stable  compiler.  APSE  change  every  4  months.  New 
tool  rate  1  per  month. 

[  ]  Stable  compiler  capable  of  tasking.  APSE  change 
every  6  months.  New  tool  rate  1  per  quarter. 

[  ]  Stable  and  fully  capable  compiler.  APSE  change 
every  6  months.  New  tool  rate  1  per  6  months. 

2.  Target  Computer  Configuration  (Complete  this  section  only  if 
the  development  system  differs  from  the  target.) 

a.  Rate  the  virtual  machine  volatility  of  the  target  system, 
based  on  number  of  major/minor  changes: 

[  ]  12  months  (major)/l  month  (minor) 

[  ]  6  months/ 2  weeks 

[  j  2  months/ 1  week 
[  ]  2  weeks/2  days 

b.  Identify  the  system  architecture: 

[  ]  Centralized  (single  processor) 

[  ]  Tightly-coupled  (multiple  processor) 

[  ]  Loosely-coupled  (multiple  processor) 

[  ]  Functional  processors  communicating  via  a  bus 
[  ]  Distributed  (centralized  database) 

[  ]  Distributed  (distributed  database) 

c.  Rehosting  Impact  (effort  to  convert  from  host  to  target 
system) 

[  ]  None 

[  ]  Minor  language  and/or  system  change 
[  ]  Major  language  or  system  change 
[  ]  Major  language  and  system  change 


I 
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PROJECT  QUESTIONNAIRE 


COMPUTER  SYSTEM,  cont'd. 


3 .  Performance  Requirements 

a.  Main  Storage  Constraint.  Main  storage  refers  to  direct 
random  access  storage  such  as  core,  integrated-circuit,  or 
plated-wire  storage;  it  excludes  such  devices  as  drums, 
disks,  tapes,  or  bubble  storage.  Select  the  percentage 
which  best  reflects  the  percentage  of  main  storage  expected 
,to  be  used  by  the  subsystem  and  any  other  subsystems 
consuming  the  main  storage  resources. 

[  ]  at  most  50% 

[  ]  70%  [  ]  85%  [  ]  95% 

b.  Overall  Hardware  Constraints.  Overall  hardware  refers  to 
processor  memory,  speed,  and  throughput 

[  ]  Close  to  100%  utilization 

[  ]  Difficult  hardware  capacity  limitations  (75%  to 
90%) 

[  ]  Average  hardware  capacity  limitations  (60%  to  75%) 

[  ]  No  hardware  capacity  limitations  (60%  to  75%) 

c.  Execution  Time  Constraints.  Select  the  percentage  which 
best  reflects  the  percentage  of  available  execution  time 
expected  to  be  used  by  the  subsystem  and  any  other 
subsystems  consuming  the  execution  time  resource. 

[  ]  at  most  50% 

[  ]  70%  [  ]  85%  t  ]  95% 

d.  Interactive  Response  Time 

[  ]  subsecond 

[  ]  1-5  seconds 

[  ]  5-10  seconds 

[  ]  greater  than  10  seconds 

[  ]  Response  time  is  not  a  factor 

4 .  Microprocessor  Code 

a.  Percentage  of  software  functions  that  are  to  be  implemented 
in  firmware;  _  % 
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PROJECT  QUESTIONNAIRE 
DEVELOPMENT  ENVIRONMENT 


1.  Project  Organization 

a.  Check  all  of  the  company  organizations  supported  by  the 
software  organization 


[ 

] 

Systems  engineering 

[ 

3 

User  department 

[ 

] 

Marketing 

[ 

3 

Software  development 

c 

] 

Program  management 

[ 

3 

Software  test 

[ 

3 

Configuration  management 

[ 

3 

Quality  assurance 

[ 

3 

Data  management 

[ 

3 

Independent  V&V 

[ 

3 

Database  administration 

[ 

3 

Other 

b.  Number  of  organizations  within  the  company  significantly 

involved  during  the  software  development:  _ 

c.  Organizational  Interface  Complexity 

[  ]  Single  customer,  single  interface 
[  ]  Single  customer  collocated  with  developer 
[  ]  Multiple  Internal  interface  single  external 
interface 

[  ]  Multiple  internal  and  external  interfaces 
[  ]  Multiple  interfaces  geographically  distributed 

d.  Multiple  site  development 

[  ]  Single  site  and  single  organization 
[  ]  Single  site  and  multiple  organizations 
[  ]  Multiple  sites,  same  general  location 
[  ]  Multiple  sites,  located  50  miles  or  more  apart 

e.  Total  size  (in  100s)  of  all  involved  organizations,  not 

just  project  personnel,  involved  in  the  project: _ 

f.  Number  of  locations  at  which  software  is  developed  (from  1 

to  100) : _ 

g.  Number  of  company  locations  that  must  be  travelled  to  for 

project  work: _ 

h.  Number  of  customer  locations: 


1 


I 

i 

j 

I 

I 
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PROJECT  QUESTIONNAIRE 
DEVELOPMENT  ENVIRONMENT 


i.  Select  the  commuting  rate  that  characterizes  the  overall 
commuting  rate  of  project  personnel: 

[  ]  Normal  (less  than  1  hour  each  way) 

[  ]  Significant  (more  than  1  hour  each  way) 

[  ]  Physical  relocation  of  project  team  for  part  of  the 
work 

j .  Number  of  physical  personnel  moves  required  between 

locations  or  departments: _ 

2 .  Computer  Resources 

a.  Select  the  percentage  of  time  that  the  development  computer 
is  available  for  use  on  this  project: 

[  ]  10%  [  ]  40%  [  ]  70%  [  ]  100%  (fully  dedicated) 

b.  Computer  terminal  allocation  to  development  team: 

[  ]  1  terminal/person 

[  ]  1  terminal/2  persons 

[  ]  1  terminal/>  2  persons 

c.  Select  the  average  time  required  to  log  on,  edit,  compile, 
link,  execute,  and  receive  a  hardcopy 

[  ]  <  4  hours  [  ]  4-12  hours  [  ]  >  12  hours 

3.  Office  Facilities 

a.  Software  Development  Office  Facilities 

[  ]  Individual  Offices 
[  ]  2-person  offices 
[  ]  Multi-employee  offices 
[  ]  Open/no  private  offices 
[  ]  Cramped 

b.  Security  Requirements 

[  ]  Unclassified 
[  ]  Uncldssif ied/Secure 
[  ]  Classified 

[  ]  Classified/physically  Secure 
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PROJECT  QUESTIONNAIRE 


DEVELOPMENT  ENVIRONMENT,  cont ' d . 

4.  System  User  Profile 

a.  Does  the  user  already  perform!  the  proposed  function? 

[  ]  No  [  ]  Yes,  manually  [  ]  Yes,  automated 

b.  User's  experience  with  related  applications: 

[  ]  Low  [  ]  Average  [  ]  High 

c.  How  will  the  user's  data  processing  knowledge  impact  the 
project? 

[  ]  Help  [  ]  Hinder 
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AFffltQEX  H 
nXUECT  DE9CSIPnCN5 


This  appendix  cxDntains  project  descriptions  of  the  eight  projects 
targeted  in  the  test  case  study.  The  information  is  presented 
according  to  the  following  outline: 


I.  Project  Description 
Application  Type 
Conplexity 
Testing  Required 
Displays 

Required  Reliability 


II.  Software  Size 

ASIDC 

New 

Reused 

Counting  Convention 

Other 

New 
Reused 
%  for  Reuse 

III.  Development  Process 

Methodologies 
Requirements 
Tools  Available 
Standards  Used 

IV.  Ccmputer  System 

Type 

Wbr3cstation  Types 
Hardware  Developed  in  Parallel 
Target  vs.  Host 
Interactive  Response  Time 
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HCOBCT  1 


APPLICAnCN  TYPE:  Oaranand  emd  Control 

OCMPUXITY:  Sane  Difficult  or  Corplex  Calculations 
Tasking  and  Generics  Used 

TESTING  REQUIRED:  Normal  Testing  by  Developer 

DISPLAYS:  User-Friendly 

Menu  Driven  Di^lays 

REQUIRED  RELTABILnY:  Speciad.  Badop  Considerations  to  Ensure  no 

Failure 

SQFTVIARE  SIZE 

SIDC:  50,000  New  Ada 

COUNTING  CX»lVENnc»l:  Terminal  Semicolons 

%  PDR  REUSE:  10%  Packaged  for  Reuse  in  any  Other  Application 

DEVELOPMENT  PROCESS 

METHODOLOGIES:  Prototyping 

Incremental  Develc^xnent 

C^ject  Oriented  Design  Plus  Structured  Aralysis 
Continuous  Integration  via  Pactege  Specs 
Monthly  Project  Reviews 

REQUIRE11ENTS:  Fairly  Cotplete  System  Requirements 

Occasional  Moderate  Requirement  Changes 

TOOLS  AVAILABLE:  Basic  Ada  Language  Tools 
Documentation  Tools 

STANDARDS  USED:  COtimercial  Standards 
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TYPE:  DistritFited  (Distributed  Database) 

VOIKSTATICN  TYPES:  2 
HARDWARE  DEVEIDFED  IN  PARALLEL:  No 
TARGET  VS.  HOST:  Same  Macdiine,  No  Change  Required 
INIERACnVE  RESPONSE  TIME:  1-5  Seconds 
PROJECT  STAFFING 
RATING: 

MANAGERS  -  Above  Average  (75%) 

ANALYSTS  -  Above  Average  (75%) 

PROGRAMMERS  -  Above  Average  (75%) 

TEAM  CXMPOSrncW:  Even  Mixture  of  Experts  and  New  Hires. 

All  Have  Had  Seme  Background  in  the  Ada 
language. 

YEARS  EXPERIENCE  WTIH  MACHINE:  2  Years 
YEARS  EXPERIENCE  VTCTO  ADA:  1  Year 
YEARS  EXPERIENCE  WITH  METHODOLOGY:  1  Year 
EXPEEILENCE  WITH  MODP:  Reasonably  Experienced  with  Seine 
TYPE  OF  TEAMS  USED:  Participatory  Teams 
TYPE  OF  EROJECT  FOR  STAFF:  Normal  New  Project 
STAFF  AVAIIABILITY;  "  - 
DEVEIDFMENT  ENVIRCMIEM' 

NUMBER  OF  SITES  DEVELOPMENT  OCCURRED  ON:  1 

PERCENT  OF  OCMEUTER  RESOURCES  AVAILABLE :  100%  (Fully  Dedicated) 

RATIO  OF  TEFMINALS  TO  PERSCMJEL:  1  Per  Person 
SECURITY  REQUIREMElTrS:  Unclassified 
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PROJECT  DESCRIPTION 

APPLTCATIC*!  TYPE:  Command  and  Control 

CCMPIEXITY:  Difficailt  Nested  logic, 

Recti -Time  Processing 

TESTING  REQUIRED:  Normal 

DISPLAYS:  Interactive  with  Pointing  Devices 

REQUIRED  RELIABILITY:  None 

SOFTWARE  SIZE 

ASLDC:  35,000  New  Ada 

OTHER:  42,000  New  Assembly 

38,000  Reused  Assembly 

COUNTING  caWENITCW:  Terminal  Semicolons 

%  FOR  REUSE:  10%  Package  for  Reuse  in  any  other  application 

DEVELOPMENT  PROCESS 

METHODOLOGIES :  Evolutionary  Development 

Object  Oriented  Design  with  Structured  Analysis 
Design  and  Code  Walkthroughs 
Phased  Builds 

REQUIREMENrs:  Fairly  Ccnplete  Understanding  of  Requirements 
Many  Major  Changes 

TOOIS  AVAILABLE:  Beisic  Ada  Language  Tools 
Document  and  Design  Tools 

STANDARDS  USED:  MIL-STD  1679  483  188 

MII/-STD  482  490 
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CCMFUTER  SYSTEM 


TYPE;  Distributed 
WDRKSTATICW  TYPES:  1 
HARDWARE  DEVELOPED  IN  PARALLEL:  Yes 
TARGET  VS.  HOST:  Major  Language  or  Sj^tem  Change 
interactive  RESPC^ISE  TIME:  1-5  Seconds 
PROJECT  STAFFING 
RATING: 

MANAGERS  -  Not  Available 
ANALYSTS  -  Average 
PROGRAMMERS  -  Average 

TEAM  OCMPOSmcW:  Even  Mixture  of  Esgserts  and  New  Hires 
YEARS  EXPERIENCE  WITH  MACHINE:  12  Years 
YEARS  EXPERIENCE  WITH  ADA;  2  Years 
YEARS  EXPERIENCE  WITH  METHODOLOGY:  0.5  Years 
EXFERIEInCE  with  MODP:  Reasonably  Experienced  with  Some 
TYPE  OF  TEAMS  USED:  BAmctioned  Teams  with  Leader 
TYPE  OF  PROJECT  FOR  STAFF;  Normal  New  Develcproent 
STAFF  AVAIIABILLTY;  100% 

DEVELOPMENT  ENVIRONMENTS 

NUMBER  OF  SITES  DEVELOPMENT  OCCURRED  ON;  3 
PERCENT  OF  OCMPUTER  RESOUPKES  AVAILABLE:  100% 

RATIO  OF  TERMINAIS  TO  PERSC^JNEL;  1 
SECURITY  REQUIREMENrS;  Unclassified 

Additional  20  Months  Due  to  Having  to  Change  Ada  Cortpilers,  Late 
Carpiler  Deliveries,  Late  Target  Ccsrputer  Memory,  Difficult 
Integration 
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HOJBCT  3 


PROJECT  CESCRIPnC^ 

APPLICSVnai  TYPE:  Environment/Tools 

Produc±ion  CJenter,  Contracted  Software 

OCMPIEXnY:  Routine  Nesting,  Minimal  Interface  with  Operating 
System,  Standard  I/O 

TESTING  REQUIRED:  Normal  Testing  by  Developer 
DISPLAYS:  Sinple  Iiputs/Outputs 
REQUIRED  RELIABIUTY:  None 


SOFIViftRE  SIZE 

ASIOC:  13,600  New  Ada 
5040  Reused  Ada 

OCUMTING  OC^A/ENFION:  Termin2d  Semioolons 

%  FOR  REUSE:  10%  to  be  packaged  for  reuse 

DEVEIOFMENT  PROCESS 

MEIHODOIOGIES:  Object  Oriented 
Ada  H)L 

Bottom  Up  Develc^xnent 
Structured  Analysis 

Chief  Programmer  for  Team  Develc^ment  Strategies 
Code  Walkthroughs 

REtyJIREMENIS;  Fairly  Ccnplete  Definition  and  Understanding 
System  Requirements 
Frequent  Noncriticad.  Changes 

TOOIS  AVAILABLE:  Requirements  Analysis 

Design  Documentation  Generation 

STANDARDS  USED:  MIL-STD  1679 
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TYPE:  Distributed 


WORKSTMTOJ  lYPES:  1 
HARDWARE  DEVEIDPED  IN  PARALLEL:  Yes 
TARGET  VS.  HOST:  None  Required 
IMEERACnVE  RESPONSE  TIME:  1-5  Seconds 
PROiECT  STAFFING 
RATING: 

MANAGEE^S  -  Not  Available 
ANALYSTS  -  Average 
PROGRAMMERS  -  Above  Average 

TEAM  OCMPOSrnc^E:  Team  Ranged  from  a  Master's  Experience,  in 

Ccnputer  Science  plus  4  Yecu::s  Job  Experience 

YEARS  EXPERIENCE  WITH  MACHINE:  1  Year 

YEARS  EXPERIENCE  WITH  ADA:  2  Years 

YEARS  EXPERIENCE  WITH  METHODOIOGY:  0  Years 

EXPERIENCE  WITH  MODP: 

TYPE  OF  TEAMS  USED:  Programming  Teams/Chief  Programmer 
TYPE  OF  roOJECT  FOR  STAFF:  Normal  New  Product 
STAFF  AVAIIABILITY:  70% 

DEVEIOFMENT  ENVIRCMgNTS 

NUMBER  OF  SITES  DEVELOEMENT  OCCURRED  OI:  1 
PERCENT  OF  COIRITER  RESOURCES  AVAIIABLE:  100% 

RATIO  OF  TERMINALS  TO  PERSCMJEL;  1  Per  Person 
SECURITY  REQUIREMENIS:  Unclassified 
EXTRA:  Three  Month  Hiatus 
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CSOOECr  4 


PROJECT  DESCRIPnOJ 

APPLICATIW  TYPE:  Envirorunental/Tools  Prototype 

Production  Center,  Internally  Develcped 

OCMPTEXITY:  Routine  Nesting,  Kinimal  Interface  with  Operating 
System,  Standard  I/O 

TESTING  REQUIRED:  Normal  Testing  by  Developer 
DISPLAYS:  Sijnple  Irputs/CXitputs 
REQUIRED  RELIABILITY:  None 
SOFTWARE  SIZE 


ASIOC:  480,000  New  Ada 
COUNTING  CCWVEMTCN:  Terminal  Semicolons 
%  FOR  REUSE:  10%  Packaged  for  Reuse 
DEVEiORgJn'  PRorryw 

METHODOIOGIES:  Object  Oriented  Design 
Rapid  Prototyping 

Independent  Teams  working  in  Parallel 
Ada  PDL 

Incremental  Develc^mient 

Ccffnpilable  Package  Specs  by  PCR 

**  Very  Familieur  with  Ada  Design  Practices 

REQUIREMENTS:  Fairly  Ccnplete  Requirements 

Noncritical  Requirement  Changes 

TOOIS  AVAIIABIE:  Document  Generator 
APSE 

STANDARDS  USED:  MIL-STD  1815 

Not  Much  Documentation  Required  Since  Prototype 
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CXagUTER  SYSTEM 


TYPE:  Distxibuted 
WDPKSTATICN  TYPES:  1 
HARDWARE  DEVELOPED  IN  PARALLEL:  No 
TARGET  VS.  HOST:  No  Change 
INTERACTIVE  RESPC»ISE  TIME:  1-5  Seconds 

roOJECT  STAFFING 
RATING: 

MANAGERS  -  98%  Tcp  Notch 
ANALYSTS  -  98%  Top  Notch 
PROGRAMMERS  -  98%  Tcp  Notch 

TEAM  OCMPOSmoi:  **  All  Experts 

Team  Ccstposed  of  Ada  Experts 

YEARS  EXPERIENCE  WITH  MACHINE:  5  Years 

YEARS  EXPERIENCE  WITH  ADA:  5  Years 

YEARS  EXPERIENCE  WITH  MEIHODOLDGY:  5  Years 

EXPERIENCE  WITH  MODP:  Reasonably  Experienced 

TYPE  OF  TEAMS  USED:  Interdisciplinary  Teams 

TYPE  OF  PROJECT  FOR  STAFF:  Normal  Project 

STAFF  AVAIIABILTTY:  100% 

DEVELOPMENT  ENVIRONMENrS 

NUMBER  OF  SITES  DEVELOPMENT  OCCURRED  CN:  1 

PERCENT  OF  COMPUTER  RESOURCES  AVAIIABIE:  100% 

RATIO  OF  TERMINAIS  TO  PERSCSJNEL:  1 

SECURITY  RBQUIREMENIS:  Unclassified 
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PROJECT  EgSCRIPnON 

APPLICATiai  TYPE;  Environmerit/Tools 

CXMPLEXITY:  Difficult  Hi^Hy  Nested  Logic,  Real-Time  Processing 

TESTING  REQUIRED:  Normal  Testing  by  Developer 

DISPLAYS:  User-Frierdly,  Menu  Driven 
Graphics  Oriented 

REQUIRED  RELIABILITY:  None 

OTHER: 


SOFLWRRE  SIZE 


ASIXX::  136,000  New  Ada 

CXXJNTING  OC»lVENnc»l:  Fhysiceil  lines  (includes  blank  lines 

and  ccaraents) 

OIHER:  None 

%  FOR  REUSE;  10% 

DEVELOFMENT  PROCESS 

METHODOLOGIES :  Incremental  Development 
Phased  Builds 
Rapid  Prototyping 
Pilot  Develc^xnent 
Small  Up-front  Design  Teams 
CXjject-Oriented  Methods 
Structured  Ancilysis 
Design  and  Code  Wal3cthrou^is 

REC?JIREMEI/IS:  Questionable  Definition  and  Understanding  of 
System  Requirements 

Ooccisional  Moderate  Changes  to  Software 
Requirements 

TOOLS  AVAILABLE;  Basic  Ada  Language  Tools 

Design  Documentatican  Generator 
GrajAiics  Generator 

STANDARDS  USED:  MIL  STD  2167A 
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I 


OCMRJTER  SYSTEM 

TYPE:  Rmctional  Processors  Ccmnunicating  via  a  Bus 

WORKSTATiai  TYPES:  3 

HARDWARE  DEVELOPED  IN  PARALLEL:  No 

TARGET  VS.  HOST:  Major  Language  or  System  OTange  Required 
for  Rehosting 

INTERACTIVE  RESPC»/SE  TIME:  1-5  Seconds 
PROJECT  STAFFING 
RATING: 

MANAGERS  -  Top  90% 

ANALYSIS  -  Above  Average 
PROGRAMMERS  -  Above  Average 

TEAM  OCMPOSmON:  Experience  Ada  Personnel 

YEARS  EXPERIENCE  WITH  MACHINE:  5  Years 

YEARS  EXPERIENCE  WITH  ADA:  3  Years 

YEARS  EXPERIENCE  WITH  METHODOLOGY:  3  Years 

EXPERIENCE  WITH  MODP:  Reasonably  Experienced  in  rfost 

TYPE  OF  USED:  Design,  Programming,  Participatory 

Interdisciplinary  Teams 

TYPE  OF  PROJECT  FOR  STAFF:  Normal  Project 

STAFF  AVAILABILITY:  75% 

nEVKTOP^[EN^  ENVERONMENIS 

NUMBER  OF  SITES  DEVELOPMENT  OCCURRED  ON:  1 

PERGENT  OF  COMPUTER  RESOURCES  AVAILABLE: :  100% 

RATIO  OF  TERMINAIS  TO  PERSONNEL;  1  Terminal/Person 

SECURITY  RBQUIREMENrS;  Uncleissified/Secure 

MICROCODE:  50% 
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PROJECT  DESCJOIPnC^ 

APPLTCMTCN  TYPE:  Avionics  (Guidance  and  Control) 

CCMPIEXITY:  Difficult  Hi^jly  Nested  Logic, 

Real-time  Processing 

TESTING  REQUIRED:  Normal  Testing  by  Developer 
DISPLAYS:  None 
REQUIRED  RELIABILITY:  None 
SOFTWARE  SIZE 

ASIOC:  19,200  New  Ada,  7,200  Reused 

OTHER:  4800  New  Assembly 

600  Reused  Assembly 

COUNTING  QONVENITCW:  TerminaLL  Semicolons 
%  FOR  REUSE:  10% 

DEVELOPMENT  PROCESS 

METHODOLOGIES :  Incremental  Develcptnent 
Phased  Builds 

Small-Up-Front  Design  Team 
More  Time  Spent  in  Design,  Less  Time  in  Code 
Integration  and  Testing 
Structured  Analysis 

Continuous  Integration  via  Package  Specs 
Conpilable  Package  Specs  by  PCR 

REQUIREHEMrS:  Fairly  Ccnplete  Definition  and  Understanding  of 
System  Requirements 

Many  Major  Changes  Occurred  with  Software 
Requirements 

TOOLS  AVAILABLE:  Basic  Ada  language  Tools 

No  Documentation  or  Design  Tools  Available 

STANDARDS  USED:  MIIr-STD  1679,  483,  490 
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CXagUTER  SYSTEM 


TYFE;  Centralized  (Single  Processor) 

V-JORKSTATIOJ  TYPES:  1 

HARDWARE  DEVELOPED  IN  PARALLEL:  None 

TARGET  VS.  HOST:  Major  Language  or  System  Change  Required  for 
Rehosting 

INTERACnVE  RESPCWSE  TIME:  Not  Applicable 
ERQJECT  STAFFING 
RATING: 

MANAGEES  -  Top  Percent  (Excellent) 

ANALYSTS  -  Above  Average 
PROGRAMMERS  -  Above  Average 

TEAM  O»lP0Srnc*T:  Experts  and  Some  New  Hires 

YEARS  EXPERIENCE  WITH  MACHINE:  0  Years 

YEARS  EXPERIENCE  WITH  ADA:  0  Years 

YEARS  EXPERIENCE  WITH  MEIHODOLOGY:  7  Years 

EXPERIENCE  WITH  MODP:  Reeisonably  Experienced  in  Some  MODP 

TYPE  OF  TEAMS  USED:  Participatory  Teams 

TYPE  OF  FROJECT  FOR  STAFF:  Normal  New  Project 

STAFF  AVAILABILITY:  100% 

PEVEIOFMENT  ENVERONMENTS 

NUMBER  OF  SITES  DEVEDOIWENr  OCCURRED  (»I:  1 

PERCENT  OF  COCUTER  RESOURCES  AVAILABLE:  100% 

RATIO  OF  TEFMINALS  TO  PEE^SCWNEL:  1  Per  Person 

SECURITY  REQUIREMENIS;  Unclassified 
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PROJECT  raSCRIPnC^T 

APPLICATION  TYPE:  Conmand  and  Control 

OCMPIZXriY:  Difficult  Hi<^y  Nested  Logic 
Reed-Time  Processing 

TESTING  REQUIRED:  Normal  Testing  by  Developer 

DISPLAYS:  Mostly  Sinple  Displays  but  a  Smedl  Peirt  is 
Graphics  Oriented 

REQUIRED  REUABIUTY:  None 

SOnWARE  SIZE 

ASIOC:  69,160 

OTHER:  None 

OOUNITNG  acWENTTCK:  Terminal  Semicolons 
%  FOR  REUSE:  10%,  of  software  to  be  packaged  for  reuse 
DEVELOFMENT  PROCESS 

METHODOLOGIES:  Incremental  Development 

Small  Up  Firont  Design  Teams 
More  Time  Spent  in  Design,  Less  Time  in  Code, 
Integration  and  Testing 
Structured  Programming 
continuous  Integration  via  Package  Specs 
Code  Walkthroucfis  and  Inspections 
Design  Walkthroui^  and  Inspections 

RBQUIREMENIS:  Questionable  Definition  of  System  Requirements 
Ooccisioncd  or  Moderate  Software  Requirement 
Changes 

TOOLS  AVAIIABIZ:  Basic  Ada  Language  Tools 
Documentation  tools 

STANDARDS  USED:  Tailored  MIL-STD  480,  483,  490,  1521 
MIL-S-52779 


H-14 


OCMRJTER  SYSTEM 

TYPE:  Distributed  System  (Distributed  Database) 
WDRKSTATICN  TYPES:  0  Workstations 
HARDWAPE  DEVEIDPED  IN  PARALLEL:  Yes 
TARGET  VS.  HOST:  Same  Machine,  NO  Change  Required 
INTERACTIVE  RESPONSE  TIME:  1-5  Seconds 
FRQJECT  STAFFING 
RATING: 


MANAGERS  -  Belcw  Average  (35%) 

ANALYSTS  -  Average  (55%) 

FROGRAMffiRS  -  Average  (55%) 

TEAM  OCMECSmc^I:  Even  Mixtaore  of  Experts  and  New  Hires 

YEARS  EXPERIENCE  WITH  MACHINE:  0  Years 

YEARS  EXPERIENCE  WITH  ADA:  0  Years 

YEARS  EXPERIENCE  WITH  METHODOLOGY:  0  Years 

ESGERIENCE  WITH  MODP:  Reasonably  Experienced  in  Scsne 

TYPE  OF  TEAMS  USED:  Design  and  Programming  Teams 

TYPE  OF  EE^QJECT  FDR  STAFF:  Although  Ccnpany  had  a  lot  of 

Experience  in  this  field,  for  this 
Division  it  was  First  of  its  Kind 

STAFF  AVAIIABILITY:  100% 

DEVEIOFMENT  ENVIRCTiMENrS 

NUMBER  OF  SITES  DEVEIOFMENT  OCCURRED  CW:  1 
PERCENT  OF  OCMTUTER  RESOURCES  AVAILABIE:  80% 

RATIO  OF  TEFMINAIS  TO  PERSONNEL:  1  Per  Person 
SECURITY  REQUIREMENIS:  Unclassified 
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PROJECT  raSCRIPnON 

APPLICATIC*!  TYPE:  Oanmand  amd  Control 

Corponent  within  a  System 

OCMPI£xnY:  Conplex  Dynamic  Resource  Allocation 
Multiple  ExD^jtion  Handlers 

TESTING  REQUIRED:  Normal  resting  By  Developer 

DISPLAYS:  Pressure  Sensitive  Devices 

required  ret  I  ABILITY:  None 

SOFIViaRE  SIZE 

ASIOC:  16,470 

OTHER:  1,830  Assembly 

OOUMITNG  OCWVENTICN:  TermincLl  Semicolons 

%  FOR  REUSE:  15%  of  Software  Packaged  for  Reuse 

DEVELOPMENT  PROCESS 

METHODOLOGIES;  Waterfall  Development 
Phased  Builds 

Object  Oriented  Methods  Plus  Structured 
Programming 

Monthly  Project  Reviews 
Exo^jtion  Handlers 
Ccjtpilable  Package  Specs  by  PER 

REQUIREMENrS;  Fairly  Ccnplete  System  Requirements 

Many  Major  Software  Requirements  Chcinges 


TOOLS  AVAILABLE:  Basic  Ada  Language  Tools 
No  Documentation  Tools 

STANDARDS  USED:  Tailored  MLL-STD  1679A 
MLlr-S-52779A 
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TYIE:  Mainframe  with  ASCII  Terminal 

Functioned  Processors  Ccitinunicating  via  Bus 

WDRKSTATI(»T  TYPES:  0 

HARDWARE  DEVELOPED  IN  PARALLEL:  Yes 

TARGET  VS.  HOST:  Different  Machines 

Major  Language  and  System  Change  Required 

INTERACTIVE  RESPONSE  TIME:  Subsecond 

RRCJECT  STAFFING 

RATING: 

MANAGEE^S  -  Above  Average  (75%) 

ANALYSTS  -  Above  Average  ^5%) 

PROGRAMMERS  -  Average  (55%) 

TEAM  CXUPOSmCN;  Even  Mixture  of  E:^)erts  and  New  Hires 
YEARS  EXPERIENCE  WITH  APPLICATia^:  5  Years 
YEARS  EXPERIENCE  WITH  MACHINE:  3  Years 
YEARS  EXPERIENCE  WITH  ADA:  0  Yeetrs 
YEARS  EXPERIENCE  WITH  MEIHODOIOGY:  2  Years 
EXPEEdENCE  WITH  MODP:  Beginning 
TYPE  OF  TEAMS  USED:  Design  and  Programming  Teams 
TYPE  OF  PROJECT  FOR  STAFF:  Normal  New  Project 
STAFF  AVAIIABILITY:  100% 
nRVRTQHi^ENr  ENVIRONMENIS 

NUMBER  OF  SITES  DEVELOPMENT  OCCURRED  C»l:  1 
PERCENT  OF  OCMEUTER  RESOURCES  AVAIIABIE:  70% 
ratio  of  TEFMINAIS  TO  PERSCXLNEL:  3  Per  Person 
SECURITY  RBQUIREMENrS:  Unclassified 
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